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Abstract 
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Introduction 

Cycle space theory is a basic chapter in complex analysis. Since since the 1960's its importance 
has been underlined by its role in the geometry of flag domains and applications, by means of 
double fibration transforms, to variation of Hodge structure and to the representation theory 
of semisimple Lie groups. This developed very slowly until a few of years ago when methods 
of complex analytic geometry, in particular the methods of Schubert slices, Schubert domains, 
Iwasawa domains and supporting hypersurfaces, were introduced. Early in 2002 those methods 
were used to settle a number of outstanding questions. This effectively enabled the use of double 
fibration transforms in all fiag domain situations. This has very interesting consequences for ge- 
ometric construction of representations of semisimple Lie groups, especially for the construction 
of singular representations. It also has many potential interesting consequences for automorphic 
cohomology and other aspects of variation of Hodge structure. In this article we survey the 
recent results, filling in the background as necessary, and present some new results that help to 
complete the picture. 

Part I, "Background", is an exposition of flag domains and their cycle spaces before the 
introduction of the new complex geometric methods. Section |l| recalls the very basic results on 
flag domains and compact subvarieties. Section ^ goes into the complex structure of these cycle 
spaces and describes the three basic possibilities. Section ^ describes a particular exhaustion 
function for measurable flag domains, and its consequences for cohomology vanishing theorems 
that are crucial to the double fibration transform and application to semisimple representation 
theory. The basic aspects of that double fibration transform are described in Section ^. 

Part II, "The Complex Geometric Approach", introduces the methods of Schubert slices, 
Schubert domains, Iwasawa domains and supporting hypersurfaces, and the use of Kobayashi 
hyperbolicity in this context. In order to orient the reader who is not working in complex 
analysis. Section His an introduction to the methods and ideas, and a sketch of the remainder 
of Part II. SectionTg introduces several related domains, and proves certain equivalences among 
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them, specifically ftadpt — ^ag — ■ Section ^ goes into the key notion of transversal Schubert 
varieties, explaining an enhanced duality theory and the argument that the cycle space VlwiD) 
is equal to the Schubert domain Qs (D) in all but certain exceptional cases related to hermitian 
symmetric spaces. The situation of hermitian symmetric spaces is completely described in 
Section ^. Another new element in this picture, the use of Kobayashi hyperbolicity, is described 
in Section |^. In Section ^ these ingredients are combined to describe the maximal domain of 
hyperbolicity; the result is fiAG — il/ — ^d{D) = fl^rlD), again except in certain exceptional 
cases related to hermitian symmetric spaces which are described completely in Section ^. 

Part III, "Applications and Open Problems" , applies the results of Part II to the mechanism 
of the double fibration transform, and discusses certain applications. The material on the 
double fibration transform, key parts of which are new, appears in Section Consequences 
for representations of real reductive Lie groups are discussed in Section ^2], and in Section |l^ 
there is a discussion of variation of Hodge structure and automorphic cohomology. 

Part I: Background. 



In this Part we describe the early results on the cycle space and the double fibration trans- 
form. For the most part those results are based on Lie structure theory. 



1 Flag Domains and Compact Subvarieties. 

We begin by reviewing the basic setup for flag domains and their maximal compact subvarieties 



which was presented in Wolf [W2|. We review the part of |W2| that is relevant to the theory of 
cycle spaces of flag domains. 

Let G be a complex semisimple Lie group and Q a parabolic subgroup. The compact algebraic 
homogeneous space Z = G/Q is called a complex flag manifold. Write g and q for the respective 
Lie algebras of G and Q. Then Q is the G-normalizer of q. Thus we may view Z as the set of 
G-conjugates of q. The correspondence is z ^ qz where q^ is the Lie algebra of the isotropy 
subgroup Qz of G at z. 

Let Go be a real form of G in the sense that there is a homomorphism (p : Gq ^ G such that 
(^(Go) is closed in G and dip : Qq q is an isomorphism onto a real form of g. In this paper we 
will only consider the situation where ip is an inclusion, ip : Gq ^ G, so we now assume Go C G 
and that Go is noncompact. 

Write g ^'g for complex conjugation of G over Go and of g over go • We recall some of the 
basic facts about Go-orbits on Z. 

li z E Z then qz n q7 contains a Cartan subalgebra f) of g. We may assume that f) = t), in 
other words that t) is the complexification of a Cartan subalgebra f)o = f) H go of go . There is a 
choice of positive root system A+ = A+(g, t)) such that q^ is the standard parabolic subalgebra 
q$ defined by some subset <f> C ^' where = 5'(g,f),A+) is the corresponding simple root 
system. In other words, qz = q$ where 

= {a e A I a is a linear combination of elements of 

$" = {a e S+ I a ^ $'■}, and 

q* = q'$ + q*" with q'$ = f) + ^ ga and q;^;" = ^ g_Q ■ 

It follows that Go acts on Z with only finitely many orbits; in particular there are open orbits. 
We refer to the open orbits as Hag domains. As Go-invariant open subsets of Z, the flag domains 
D <Z Z are Go-homogeneous complex manifolds. 
Measurability. 
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A flag domain D = Go{z) d Z is called measurable if it carries a Go-invariant volume 
element. This is the type of flag domain currently of most interest in representation theory. 
More precisely, the following conditions are equivalent: 
(1.2a) The orbit Go{z) is measurable. 

(1.2b) Go n Qz is the Go-centrahzer of a (compact) torus subgroup of Go . 

(1.2c) D has a Go-invariant, possibly-indefinite, Kahler metric, thus a Go-invariant 

measure obtained from the volume form of that metric. 
(1.2d) $^ = $'', and = where = q$ . 
(1.2e) n q7 is reductive, i.e. Hq^ — c\l Hql . 
(1.2f) q.nql==q^ . 

(1.2g) q is Ad (G)-conjugate to the parabolic subalgebra q'' + q" opposite to q. 

In particular, since (1.2g) is independent of choice of z, if one open Go-orbit on Z is measurable 

then all open Go-orbits are measurable. 

Condition (1.2d) holds whenever the Cartan subalgebra ()o = f) n go of go corresponds to a 
compact Cartan subgroup Hq C Go . (Here t) = f) is the Cartan subalgebra relative to which 
qz — q* •) For in that case a = ~a for every a G A(0, f)). In particular, if Gq has discrete series 
representations (so that by a result of Harish-Chandra it has a compact Cartan subgroup) then 
every open Gg-orbit on Z is measurable. Condition (1.2d) is also automatic if Q is a Borel 
subgroup of G, and more generally Condition (1.2g) provides a quick test for measurability. 
Compact subvarieties. 

We now fix z G Z such that D — Go(z) is open in Z. For convenience we suppose that z 
is the base point in Z — G/Q, so Q = Qz and q = q^ . For notational consistency with many 
papers in this area, we write L for the Levy component of Q. So -D is measurable if and 
only if Q n Go is a real form Lg of L, and in that case D = Gq/Lq . 

Fix a Cartan involution 9 of Go that stabilizes the Cartan subgroup Hq C Go , and denote its 
fixed point sets on Go and G by Kq = Gq and K ^ G^ . Then Kq is a maximal compact subgroup 
of Go and K is its complexification. L n Kq is a real form oi Ln K and Ko{z) = Kq/{L n Kq). 

As D is open we may assume f) chosen so that fl J^o is a Cartan subgroup of Kq , in 
other words so that Hq is a fundamental Cartan subgroup of Go . Use t) for the standard 
Weyl basis construction of a 6'-stable compact real form g„ C g. Then Go H G„ = Kq and 
t = (t n I) + (t n r_) + (e n r+). Thus K{z) ^ K/{K n Q) is a complex flag submanifold of Z, 
and Kq acts transitively on it. In summary. 

Lemma 1.3 K{z) = Kq{z); in particular it is a compact complex submanifold of D. 

We write Go for the compact complex submanifold Ko{z) C D. It will be the base cycle in 



a certain cycle space discussed below. The discussion leading to Lemma 1.3 shows that Go is 
both the unique i^-orbit in D that is compact and the unique i^o-orbit in D that is complex. 
This is the origin of what is known as "Matsuki duality." 

Example 1.4 Let Z be the complex projective space CP" and let Go — SU{n,l). Let 
{ei, . . . , e„+i} denote the standard basis of C""*"^ relative to which the hermitian form defining 

Go is {u,v) ~ (j2i<a<n^o.v^^ —Un+iVn+i ■ Then Go has three orbits on Z: the (open) unit 

ball B in C" inside Z, consisting of the negative definite lines, the (2n — l)-sphere S which 
is the boundary of B, consisting of the null lines, and the complement D oi BU S, consist- 
ing of the positive definite lines. D is the non-convex open Go-orbit on Z. Here Go is the 
hyperplane at infinity, complement to C" in Z. In homogeneous coordinates [z^ , . . . , z'^'^^], 
B is given by Ei^a^« l-^^^P < S is given by Ei^q^„ = D is given by 

Ei^a^n 1^1' > l^"""'!'- and Go is given by |z"+i|2 = 0. 

Later we will see Go as the base cycle in D. In this case Go is maximal among the (complex) 
subvarieties of Z contained in D. 
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2 Basic Facts on the Cycle Space. 



Basic facts about the cycle space are given in Wells & Wolf |WeW] and in Wolf |W7|. We 
review some of that material now, and briefly indicate some of the applications of cycle spaces 
to variation of Hodge structure, specifically to period matrix domains, and construction of 
automorphic cohomology classes by Poincare i^-series. Those applications, and several others, 
will be discussed in more detail in Part III below using the tools which are described in Part II. 



Definition. Let E — {g G G \ gCo = Co}. Then E' is a closed complex subgroup of G, so 
the quotient manifold := {gGo \ g G G} = G/E has a natural structure of G-homogeneous 
complex manifold. Since Co is compact and D is open, the subset {gCo \ g G G and gCo C D} 
is open in f2, and thus has a natural structure of complex manifold. The cycle space of D is 



(2.1) 



i^w{D) : topological component of Cq in {gCo \ g G G and gCo C D}. 



Thus Clw{D) has a natural structure of complex manifold. 
Hermitian trichotomy. 

In order to understand the structure of Z, D and Qw{D) we may assume that Go is simple, 
because Go is local direct product of simple groups, and Z, D and QwiD) break up as global 
direct products along the local direct product decomposition of Gq . From this point on Go is 
simple unless we say otherwise. 

Since Qq is simple and e contains 6, there are four possibilities, one trivial. The trivial one is 
the case e = g, in other words the case where Go acts transitively on Z, and Clw{D) is reduced 
to a single point. There are just a few possibilities for this (Wolf | WS]). From now on we ignore 
this trivial case and concentrate on the other three: 

1. Hermitian holomorphic case. Gq/Kq is a bounded symmetric domain B, we have the 
usual = p++6 + p_ , and e is one of 6 + p± . In this case D is measurable, say D = Gq/Lq, 
and there is a holomorphic double fibrationQ 

Go/(£o n i^o) 



D 



B 



In other words, the two projections are simultaneously holomorphic for some choice be- 
tween B and the complex conjugate structure B and some choice of invariant complex 
structure on Go/(io H Kq). In this case Go/(io H Kq) is the incidence space T{D) that 
we'll meet later, and nwiD) is B or B; see ^ or Wolf-Zierau |WZl[. 

Hermitian non-holomorphic case. Gq/Kq is a bounded symmetric domain B and 
e ~ t. In this case we cannot adjust invariant complex structures so that the double 
fibration indicated just above will be holomorphic. Here it was recently proved that flwj P) 
is biholomorphic to B xB. See ^ or [|WZl| , and Huckleberry-Wolf |HW3| or jwzj j. 

Generic (or non-hermitian) case. Gq/Kq does not have a Go-invariant complex 
structure. Then t is a maximal subalgebra of g. In this case K is the identity identity 
component of E, Q is an affine homogeneous space, and we will describe the structure of 
flw{D) in that context. The precise structure was worked out only very recently in |HW3| 
and Fels-Huckleberry | FH ] . 



^ By holomorphic fibration we mean a holomorphically locally trivial fiber space, essentially a holomorphic fiber 
bundle except perhaps lacking a complex structure group. 
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In the rest of this article, we describe complex geometric methods that lead to the developments 
indicated above, and to other applications and developments through the use of double fibra- 
tion transforms. The new developments include aspects of the theory of holomorphic double 
fibration transforms themselves (Section |ri|). The areas of application include aspects of the 
representation theory of semisimple Lie groups (Section ^|) and variation of Hodge structure 
(Section 



3 The Exhaustion Function and [q + 1)— Completeness. 

Measurable open orbits D = Go{z) C Z carry an especially useful real analytic exhaustion 
function ip : D M. whose Levi form C{ip) has at least n — q positive eigenvalues at every 
point of D, where n = dime D and q — dime Co ■ Thus ip is strongly g-pseudoconvex and D is 



(q + l)-complete. In this section we review that development from [Sl|, [ WeW and |3W|, and 



then we indicate applications [W7| to cohomology over D and to the Stein property of flwiD). 



The exhaustion function (p : D —^M. was first described in Schmid's thesis in the setting 
where Gg has a compact Cartan subgroup and Z = G/B where B \s a. Borel subgroup of G. 
The 6'-stable real form Gu of G acts transitively on Z. The canonical line bundle Z, 
and the (dual) anticanonical line bundle Z , are G„-homogeneous and have G„-invariant 

metrics. Let hu denote the Gu-invariant hermitian metric on Z. In this setting the 

isotropy subgroup Lq of Go at a point z e D of the open orbit is just a compact Cartan 
subgroup, so the anticanonical bundle KJj — > D has a Go-invariant hermitian metric Hq . Then 
one has the (real analytic) positive function ip = log ho/ on D. If g{z) G bd(D) then 
Ad(g)([ + q_) + Ad{g){l + q_) ^ g, and it follows that ip goes to infinity as one approaches 
g{z) from the interior of D. From this one sees that </? is an exhaustion function for D. Root 
space considerations allow one to compute >/— 1991og ho and V— 199 log hu and see the Levi 
form C{(p) explicitly. It follows immediately that C(p) has at least n — q positive eigenvalues at 
every point of D. 



Somewhat later. Wells and Wolf WeW ] noted that Schmid's argument could be adapted to 
the more general setting where the only requirement is that the isotropy subgroup Lq of Go 
at a point z E D is compact. Somewhat after that, Schmid and Wolf [^W|] further adapted 
the argument to the (even more general) situation where D is a measurable open Gg-orbit in a 
complex flag manifold Z = G/Q. Thus every measurable open orbit D is {q-\- l)-complete. 

Cohomology vanishing theorems. 



The theorem of Andreotti and Grauert | AnG says that if a complex manifold D is (q + \) 



complete, and if 5 ^ £> is a coherent analytic sheaf, then the cohomologies H^{D;S) — for 
for all r > q. 

Since our measurable open Gp-orbit D is ((7+l)-complete, we have the vanishing H^{D; 0{E)) 
for r > q, for every holomorphic vector bundle E _D. On the other hand, if E ^ is a 



(sufficiently) negative bundle |GrS the methods based on the Bott-Borel-Weil Theorem show 



that H''{D; 0{E)) = for r < q. Thus, finally, 

(3.1) if E -> D is (sufficiently) negative, then H''{D; 0(E)) = for r 7^ g. 

This will be very important when we discuss double fibration transforms. 
The Stein property for cycle spaces of measurable open orbits. 



When D is a measurable open orbit. Wolf | W7| combined his extension of boundary compo- 
nent theory of bounded symmetric domains (| ]W2| , or see [ |W4[ ) with the exhaustion function 
iy9 : — > M, to prove that flwiD) is a Stein manifold. We review the argument. 
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Case: D is of hermitian holomorphic type. Here we may assume D = Goiz) and 
E = KP-, so = G/E is the compact hermitian symmetric space dual to the bounded 
symmetric domain B. Thus B C flw{D) C and il.w{D) is invariant by the action of Go 
on fl. The Go-orbit structure of il, and the closure relations among the Go-orbits, are known 
precisely in terms of partial Cayley transforms (|W2], |W4|). If O is a Go"Orbit in flwiD) , 
then it contains every open Go~orbit whose closure contains O, because D,w(D) is open in Cl. 
Some operator norm arguments show that fl\y{D) cannot contain an open orbit different from 
B. It follows that Qw{D) = B, and in particular Qiy{D) is Stein. 

Case: D is not of hermitian holomorphic type. Then E has identity component K, 
so E is reductive and ft — G/E is afhne. Define f3 : flwiD) M+ by (3{gGo) = supy^f^^ ip{g{y)). 
Since ip is an exhaustion function and the gCo are compact, one sees that P : Qw{D) ~f 
blows up at every boundary point of On/(D) . From the specific construction of (p, and a close 
look at the real analytic variety given hy dip = 0, one sees that (3 is continuous, piecewis e G" 
and plurisubharmonic. Now a modification suggested by results of Docquier and Grauert [DG| 
gives a strictly plurisubharmonic exhaustion function -tp — ip + v constructed as follows. 
Since is Stein there is a proper holomorphic embedding / : — > C^"+^ with closed image, by 
Remmert's theorem. Define iy{G) := ||/(G)|p for G G ftw{D) . Since flw{D) carries a strictly 
plurisubharmonic exhaustion function, it is Stein. 



4 Early Problems and Results on the Double Fibration 
Transform. 

We start this section with a review of some basic facts on the double fibration transform from 
I WZ2[ |. We then speciahze (initially as in |WZ2|) to the case of an open orbit D = Gq{z) C Z, 



and present some new results that clear up several open problems in that fiag domain case. 
Finally we give a quick indication of the consequences for variation of Hodge structure and for 
semisimple representation theory. Now we start with the general setup, indicate its technical 
requirements, and specialize it to our flag domain situation. Several of the problems that come 
up here are settled later in Section |ll| using methods developed in Part II below. 

Double fibration. 

Let D he a complex manifold (later it will be an open orbit of a real reductive group Go 
on a complex flag manifold Z — G/Q of its complexification ). We suppose that D fits into 
a holomorphic double Ebration, in other words that there are complex manifolds M and I{D) 
with simultaneously holomorphic fibrations: 

(4.1) 

I{D) 

V K 
D M 

(Later M will be a cycle space and T{D) will be an incidence space for points and cycles.) Given 
a coherent analytic sheaf £ ^ D we construct a coherent sheaf £' M and a transform 

(4.2) P : i7«(L>; £) H°{M: £') 

under mild conditions on (|4.lD. In fact we give several variations on the construction. This 



construction is fairly standard (see, for example, |BE], |PR1 and Q), but we need several 
results specific to the case of fiag domains. 
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Pull back. 

The first step is to pull cohomology back from D to I{D). Let ^^^{£) I{D) denote the 
inverse image sheaf. For every integer r ^ there is a natural map 

(4.3) M^--) : H^D-£) ^ {X{D)- ^i-\£)) 

given on the Cech cocycle level by /x'^''^ (c)((t) — c{p{a)) where c e Z'^{D;£) and where a — 
(wqj ■ ■ ■ ,Wr) is a simplex. For q ^ we consider the Buchdahl g-condition 

(4.4) the fiber F of a* : 1(0) D is connected and i?'^(i^; C) = for 1 ^ r ^ g - 1. 



Proposition 4.5 (See |Bu|.) Fix q ^ 0. If (4^) holds, then (|4.3| ) is an isomorphi sm for 
r ^ q — 1 and is injective for r = q. If the fibers of fi are cohomologically acyclic then (4.3) is 
an isomorphism for all r. 

As usual, Ox X denotes the structure sheaf of a complex manifold X and ©(E) X 
denotes the sheaf of germs of holomorphic sections of a holomorphic vector bundle E ^ X. Let 
H*{£) := lJ-^^{£)^fj,-i{On)^T{D) ~* ^{D) denote the pull-back sheaf. It is a coherent analytic 
sheaf of Oi(£)) -modules. If E = 0(E) for some holomorphic vector bundle E Z?, then 
^*{£) = 0{fi*{E)), where /i*(E) is the pull-back bundle. In any case, [a] t-^ [a] (g) 1 defines a 
map i : fi^^{£) fJ-*{£) which in turn specifies maps in cohomology, the coefficient morphisms 

(4.6) Zp:HP{I{Dy,fi-H£))-^ HP{I{D);fi*{£)) for p ^ 0. 

Our natural pull-back maps are the compositions j'^p^ = ip ■ ^f^^'^ of ( |4.3| ) and ( |4.6| ): 

(4.7) j^^') : HP{D;£) ^ H'p {I{D)- ti* {£)) for p ^ 0. 

li £ ~ 0{E) for some holomorphic vector bundle E ^ Z?, then fi*{£) = C'(/i*(E)), we realize 
these sheaf cohomologies as Dolbeault cohomologies, and the pull-back maps ( |4.7| ) are given by 
pulling back [uj] i-^ [/i*(w)] on the level of differential forms. 

Push—down. 

In order to push the II'^{T{D); fJ,*{£)) down to M we assume that 

(4.8) v : I{D) ^ M is a proper map and M is a Stein manifold. 



The Leray direct image sheaves TZP{p*{£)) M are coherent [ GrR | . As M is Stein 

(4.9) H'>{M;nP{£)) =0 for p ^ and g > 0. 
Thus the Leray spectral sequence collapses and gives 

(4.10) HP {I{D);fi*{£))^H'{M;nP{^i* {£))). 



Definition 4.11 The double fibration transform for the holomorphic double fibration (4.1) is 
the composition 

(4.12) P : HP{D;£) H°{M;nP{^i* {£))) 

of the maps (|j|) and ( [I.IOD . 
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In order that the double fibration transform (4.12) be useful, one wants two conditions to 
be satisfied. They are 

(4.13) P : HP{D;£) H°{M;nP{n*{£))) should be injective, and 

(4.14) there should be an explicit description of the image of P. 

Assuming ( |4.8| ), injectivity of P is equivalent to injectivity of 

jip) in 

( [4.71 ). The most general 

way to approach this is the combination of vanishing and negativity in Theorem 4.15 below, 
based on the Buchdahl conditions (4.4). 



The general (assuming (4.8)) injectivity question uses a spectral sequence argument for the 
the relative Dolbeault complex of the holomorphic fibration /i : T{D) D. See [ WZ2| for the 
details. The end result is 

Theorem 4.15 Fix q ^ 0. Suppose that the fiber F of fi : 'I{D) —fDis connected satisfies 
(4.4). Assume ( |4.8| ) that v : T{D) —> M is proper and M is Stein, say with fiber C. Let 
Q^^(E) ^(D) denote the sheaf of relative fi*E,-valued holomorphic r-forms on T{D) with 
respect to fi : T{D) — > D. Suppose that _ffP(C; ri^(E)|(7) = for p < q, and r ^ 1. Then 
P : i7« (£>;£) ijO(A/; 7^«(^*£)) is injective. 

Remark 4.16 In the cases of interest to us, £ — 0{K) for some holomorphic vector bundle 
E ^ Z?, and P has an explicit formula. The Leray derived sheaf is given by 

(4.17) 7^''(^^*(0(E))) ^ O(E^) where E^ M has fiber i/«(j/-i(C); C'(/i*(E)|^-i(c.))) at C. 
Let Lo be an E-valued (0, (7)~form on D and [uo] e iJ-|-(Z3,E) its Dolbeault class. Then 

P{[uj\) is the section of E^ M whose value P{[uj]){C) at C G M is [^*(w)|,,-i(c)]. 
In other words, 

(4.18) Pm{C) = K(a;)|,-.(c)] € i?§(M; Et). 

This is most conveniently interpreted by viewing P{\u)]){C) as the Dolbeault class of wjc", and 
by viewing C > [oj\c] as a holomorphic section of the holomorphic vector bundle E^ M . 

Flag domain case. 

Now let D — Gq{zo) be an open orbit in the complex flag manifold Z — G/Q, and M is 
replaced by the cycle space ^lw{I^)- Our double fibration (4.1) is replaced by 

(4.19) 

I{D) 
V K 

D nw{D) 

where T{D) := {{z,C) E D x ftwiD) \ z E C} is the incidence space. Given a homogeneous 
holomorphic vector bundle E — )■ D, and the num ber q = dime C'o ; the Leray derived sheaf 
involved in the double fibration transform satisfies ( 4.17 ). Here that takes the form 

(4.20) 7^«(Ai*(C'(E))) = 0{E^) where E^ ^ nw{D) has fiber H''{C:0{E\c)) at C G nw{D). 

Evidently, E^^ flwiD) is globally Go-homogeneous and infinitesimally g-homogeneous, and 
H'^{C;0{K\c)) can be calculated in any reasonable case from the Bott-Borel-Weil Theorem, 
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especially when E 
domain case. 



D is negative. Thus 7^'^(/i*(0(E))) is given explicitly by (4.20) in the flag 



Using methods of complex geometry to be described in Part II, we will see in Part III that 
Vlw{D) is a contractible Stein manifold in general, so fit Q,w{D) is holomorphically trivial, 
and that F satisfies (4.4) for all q, so the double fibration transforms are injective. Thus, in 



the flag domain case, we will have a complete answer to (4.13) and some sharp progress toward 
dTT^ ). See Section |T[ 



Now let us take a quick historical look back at the hermitian trichotomy of Section |2| In 



the hermitian holomorphic case Vlw{D) is B or B, and one knows WZ2 
and U,w{D) are contractible Stein manifolds 



Section 4] that F 
In the hermitian nonholomorphic case, where 



Q.w{ p) ^ Bx B ([|W7[ or |WZl|] , and |HW3| or |WZ3[ ), there had only been partial information 
(see I WZ2| , Theorem 6.6]) on contractibility of F. There had been essentially no information in 
the nonhermitian case. 

One more remark. In some cases one knows that ^^{D^E) is an irreducible representation 
space for a group under which all our constructions are equivariant, and one sees directly that 
P is an intertwining operator, thus zero or injective. In practice, however, we usually look for 
implications in the other directions. See Section |ll|. 



Part II: The Complex Geometric Approach. 

In this Part we describe the methods and results in complex geometry that lead to a structure 
theory for the cycle space, various associated domains, and the double fibration transform. 



5 Introduction to the Complex Geometric Approach. 



Our goal here is to explain recent results which have led to the characterization of ^w{D) as 
being equivalent to a certain universal domain Hag in but the well-understood trivial and 
the hermitian holomorphic cases as discussed in Section ^j. Without further reference we exclude 
those cases in the sequel. 

In the present section we outline the relevant results and methods in a nontechnical way. In 
the following sections we give enough details so that the reader should have no difficulty working 
through the literature. See |IIW3| and |FH| for complete details. 



Building on experience with transversal varieties gained in |IIW1|, |W9| and |II5], the Schu- 
bert domain ^s{D) was introduced in [H as a tool for understanding complex analytic properties 



of Q.w{D). The motivation for this is quite transparent in |EIS|, although there only the case of 
Go = SLn{^) was considered. 

We discuss [HSj with the benefit of hindsight and a more up to date notation. Let Gq — 
KqAqNi^ be an Iwasawa decomposition of the real form Gq. The corresponding set KAN 
corresponds to the open AA^~orbit in the spherical affine homogeneous space O := G/K and 
is a proper, Zariski open subset of G. We refer to a Borel subgroup i? C G as an Iwasawa- 
Borel subgroup of G if it contains an Iwasawa factor AqNq. Of course these are just the Borel 
subgroups which occur as the isotropy groups at points of the closed Go-orbit in G/B. Given 
an Iwasawa-Borel subgroup _B in G and a point z € Z = G/Q, the closure S = c£{0) of the 
orbit O = B.z is the associated Iwasawa-Schubert variety or Schubert cycle. Let Y denote the 
complement of the Schubert cell O £ S, i.e., Y S \ O . 

An Iwasawa-Schubert variety T = c£{0) is transversal (relative to an open Go-orbit D) if 
(1) T n D is nonempty and contained in the open S-orbit O, and (2) codimT — q = dim Go 
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and the intersection T H Co is transversal at each of its points. In this dimension n — q, the 



Iwasawa decomposition of Go imphes that Y C Z \ D (Theorem 3.7). 

It was shown in pIS| | that for an open S'L„ (E)-orbit D in a flag manifold Z of S'L„(C), and 
C G hd{n\Y{D)), there exists a transversal Schubert variety T such that YOC ^ 0. The method 



of incidence varieties ([BK],[BM|) then shows that the algebraic variety Ay := {C € fl : C HY} 
contains the polar set Hy of a nonconstant meromorphic function which is produced by the 
method of trace transform. In this context we refer to Hy as an incidence hypersurface. It 
follows immediately that ilwlD) is a Stein domain in fl, because each of its boundary points is 
contained in an analytic hypersurface which is entirely contained in its complement. 

Returning to the general case, we note that if T is Poincare dual to Co, then it is indeed a 
transversal Schubert variety. However, at least initially there is no reason to believe that, given 
C G hdiflwiD)), there exists T with C n F 7^ 0. 

Domains with supporting analytic hypersurfaces at each of their boundary points have op- 
timal character from the complex analytic viewpoint. Thus the following Schubert domain was 
introduced in Q. Let flsiD) be the connected component containing Co of the complement 
of the union of all Iwasawa-Borel invariant intersection hypersurfaces Hy which are defined by 
transversal Schubert varieties. Since Kq acts transitively on such Borel groups, the set which 
is removed is a compact family of hypersurfaces and clearly f2s(D) is a proper, open. Stein 
domain in fl which contains ilw{D). It should be emphasized that the sets Y are possibly very 
far away from the boundary of the cycle space and that the inclusion ^lw{D) C r2s(D) could 
theoretically be proper. 

In principle there could be a plethora of domains Qs{D), but experience with real forms of 
SLn{C) (see [IIW2 for the remaining cases) and classical hermitian symmetric spaces ([]Nl|], 



I WZlJ ) suggests that they might all be the same, agreeing with a domain which is defined by 



removing all Iwasawa-Borel invariant hypersurfaces from Q. 

More precisely, let flj be defined as the connected component containing Co of the com- 
plement of the union of all i3-invariant algebraic hypersurfaces in Q, where B runs over all 
Iwasawa-Borel subgroups of C. Note that, without further information, ilj could theoretically 
be empty. But in any case, using the same argument as above, it is a Stein domain and of course 
r2/ C il.s{D) for every open orbit D in every G-flag manifold Z = G/Q. In order to understand 
the relation of the cycle spaces to the universal domain ^IaGj it is natural to compare Qj and 
flAG- In fact, using the identification of ^Iag with the maximal domain of definition Qadpt of 
the adapted complex structure in the tangent bundle of the Riemannian symmetric space Go / Kq 
([ BIIII |,|Ha|; see Section ^ below), an elementary argument involving the transported norm 



function shows that ^Iag C il/ Q. 

In another guise fli had been considered and, from a completely different viewpoint the 



above inclusion had been shown for classical groups |K5]. Recently, a purely algebraic proof 



was given in |M2| 



Regarding Qj as the polar Xq (the two definitions are easily seen to be equivalent (see Q), 
but nevertheless refiect two very different aspects of the subject), Barchini proved the opposite 
inclusion [Q. 

As a consequence of considerations of cy cle sp aces associated to non-open Go-orbits, it is 
implicitly shown by case by case methods in [|GM| that ^Iag C flw{D) for classical groups and 
exceptional hermitian groups. Thus ^ag — CI fls{D) was known at this point. 



We indicate our contributions; they will be sketched in more detail below. In [HW3] we 



carried out the general program which was indicated by the naive flag arguments of [HS|: For 



every C G hd{^w{D)) there exists a transversal Schubert variety T so that Y (1 C 7^ 0; in 
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particular, flwiD) — fls{D) and thus flwiD) is a Stein domain with respect to functions in 
the image of the trace transform. 

The construction of T results from the construction of a Schubert variety Yp containing an 
arbitrarily given boundary point p € bd(-D), with codiml^ — q-\- and of course Yp C Z\D. 
This is a consequence of a sort of triality that initiates with Matsuki duality. It is of interest 



that the moment map, Morse-theoretic method for realizing this duality (|MUV],|BL|) plays a 
direct role in our considerations (see Section^). Furthermore, it is shown that, given C in the 
boundary of the cycle domain, t her e exits p e C so that Yp is indeed the F of a transversal 
Schubert variety T (see Theorem |7.6| ). So the incidence variety method along with the inclusion 
^AG C ni yields ^ag C ^w(D) = ^s{D). 

Satisfied with identifying ^Iag with Qadpt or fi/, we have actually not given its original 



definition | AkG | . We do that now because we will need its computable nature. 



The symmetric space fio ■= Gq/Kq is embedded as a totally real submanifold of half- 
dimension of = G/K as the Go-orbit of the neutral point. General principles imply that 
there are Go-invariant neighborhoods of flo in fl on which the Go-action is proper. One looks 
for a canonically defined domain for which this is the case. 

Let go = to ® Po be a Cartan decomposition set up in the usual way with respect to a 
compact real form g„ of g. Let ao C po be maximal abelian as above. For a a root of ao, 
let Ha := {A ao : a ^ ^} and define loq as the connected component containing of the 
complement in ao of the union of the Ha- Then Qag ■— Gq. exp (1^0)^0, where zq is a base point 
corresponding to Gq, is an open neighborhood of f2o a-nd is maximal with respect to the property 
that every Go-isotropy group is compact. The Go-action on ^Iag is proper (see Section^). 

The fact that the Go-action is proper indicates that an invariant metric (perhaps of canonical 
nature) is playing a role. Thus, in |Q methods were introduced to study the hyperbolicity of 
such domains. Let us recall the basic facts which are relevant for such considerations. 

The Kobayashi pseudo-metric on a complex manifold X can be defined as follows. First, 
define a disk in X as the biholomorphic image of the unit disk in the complex plane. A chain 
of disks is the union of finitely many such disks which overlap (on open subsets) to form a 
connected set. Given p,q G X consider a chain k = Ai U . . . U with p G Ai, g G A^ and 
Ai n Ai+i 7^ 0. Let Pi e Ai n Ai+1 and di be the distance from pi_i to pi computed in the 
Poincare metric of A^. Adding up these distances we obtain a number d{K) which also depends 
on the choices of the pi which are regarded as part of the data of the chain. Finally, define the 
Kobayashi pseudo-distance between p and q as the minimum of all such d{K) as k runs over all 
such chains. This defines a pseudo-distance function on X x X which, if it is nonzero for all 
p,q G X, is the Kobayashi metric In this case X is called Kobayashi hyperbolic. 

The methods introduced in Q could be regarded as leading to the group-theoretic version 
of the fact that the complement in CPm of 2m + 1 hyperplanes in general position is Kobayashi 
hyperbolic (a result of classical geometry). It is shown in H] that, e.g., fi/ is hyperbolic. These 



methods, refined in [FH|, lead to the following result (see Theorem 3.6) in the non-hermitian 
case. Let H be any Iwasawa-Borel invariant hypersurface in $7 and define ^Ih to be the connected 
component containing Go in Q of the complement of U/esifo ^i^)- other words, the definition 
of ^Ih is analogous to 05(D) and 0/ except that one initially has only a single hypersurface. 

Theorem 5.1 If Gq is not of hermitian type, then Qh is Kobayashi hyperbolic. 

Summarizing the above, if Go is not of hermitian type then, for any Iwasawa-Borel invariant 
hypersurface H in the complement of U,w{D) 

(5.2) Oag = C Ovi/(D) = Os(D) C Oi/ 
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and each of these domains is Stein and Kobayashi hyperbohc. 



The main new development in |FH] is summarized as foUows 



Theorem 5.3 Suppose that Gq is not of hermitian type. If fl is a Go-invariant, Kobayashi 
hyperbolic Stein domain that contains ^IaGj then f2 — ^ag- 



It follows that the inclusions in (5.2) are equalities. With some additional remarks to handle 
the hermitian case (which in fact is much simpler), it follows that flwiD) — ^ag in a-U but the 
well understood trivial and hermitian holomorphic cases discussed in Section ^. 



We complete these introductory remarks by outlining the basic ideas of |FH|. Roughly 
speaking, the goal of that work is to reduce to the case of SL2. This requires a rather detailed 
analysis of the Go-action on bd(il^G), in particular a detailed description of closed orbits and 



orbit closures (see the subsection on close d or bits in Section 10 below). The boundary hd{flAG) 



is not smooth, but for the purposes of [FH| it is sufficient to consider points in a tractable 
open, dense stratum hdgeni^AG) (see the subsection on genericity in Section ^ below). For 
z G hdgeni^AG) One determines a 3-dimensional simple subgroup S G G which is defined over 
M such that S.z n ^Iag is the associated domain ilAG{S) for the group S with respect to its 
noncompact real form 5*0. 

Since no difficulties are introduced by going to finite covers, it may be assumed that the 
associated affine variety ft{S) is the complement of the diagonal in CPi x CPi, i.e., the 2- 
dimensional affine quadric. Regarding it as a closed S-orbit in il, we refer to fl{S) as a Q2-slice. 

The S-action on CPi x CPi is the standard diagonal action. The intersection Vl{S) nilyiG — 
^ag{S) is a 2-dimensional polydisk A which can be regarded as being the product S+ x of 
an ^Q-invariant (1-dimensional) disk with its exterior in CPi. Due to the genericity assumption, 
the point z can be chosen to lie in x bd(S^). 



It is shown in [FH] that if Q[S) is a iSo-invariant Stein domain in Q[S) which contains 
VIag{S) and the boundary point z, then it contains the open set Vl n {B^ x CPi) = x C. It 
is straightforward to check that if a complex manifold X contains a (biholomorphic) copy of Y 
of C, then the Kobayashi distance between any two points in Y is zero. Thus the domain 0(5") 
is certainly not Kobayashi hyperbolic. 



The proof of Theorem 5.2 is then immediate, because, if is a Go-invariant, Stein domain 
which properly contains ^aGi then there is a generic boundary point z such that Q{S) n O 
contains both Q.ag{S) and z. Since ^Iag{S) is Stein and S'o-invariant, it contains the copies of 
C as above and therefore 17 contains these as well. As a consequence is not hyperbolic and 



Theorem 5.3 follows 



6 The Equivalences ^adpt = ^ag = 

Here three Gq domains are introduced from three different viewpoints. The domain ^adpt in the 
tangent bundle of the Riemannian symmetric space Oo = Gq/Kq can be defined by either metric 
or symplectic properties. The equivalence of these two ways of viewing fladpt are important for 
complex analytic considerations. 

The domains ^Iag and f2/ are defined as neighborhoods of flo in the affine homogeneous 
space ri = G/ K, where a base point xq has been chosen so that f2o = Gq.Xq. The domain Qag 
is defined from the point of view of group actions. The domain flj can be seen from several 
viewpoints. Ours is that of incidence divisors which are defined by Schubert varieties in Z of 
Iwasawa-Borel subgroups B C G. 
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Adapted complex structures. 

Beginning with the metric standpoint, let (AI,g) be a real analytic Riemannian manifold 
which for simplicity is assumed to be complete. The differential 7* of a geodesic is a map 
7* : TR TM of tangent bundles and can be viewed as an orbit of the M-action defined by 
geodesic flow. Let W act by scalar multiplication in the fibers of TM . This action extends in the 
usual way to a map M x TM — > TM. Identify TM with the complex plane C by {t, s^|t) 1— > t-\-is 
and define 7'' : C = TM TM by z = s + if s ■ 7*(t), where the multiplication comes from 
the M*-action. 

Definition 6.1 An integrable complex structure J on a starlike neighborhood of A of the 0- 
section in TM is adapted if for every geodesic 7 there is a disk A = A(0) C C such that 
7''-'|a : a ^ is holomorphic. 



The existence and uniqueness of adapted structures are proved in (|LS|; also see [Ha|). The 



uniqueness statement says: If Ji and J2 are adapted structures on A, then Ji = J2. 



The symplectic side of the picture was developed in (|GuS]; also see |Bu|). Let \std (resp 



'jJstd) be the standard 1-form (resp. 2-form) on the cotangent bundle T*M. Let Pg = TM — > M 
be the norm-function defined by the metric, i.e., pg(-) :— \\ ■ If ijjg : TM T*M denotes the 
diffeomorphism defined by the metric, then we have the forms 9g := ipli^std) and ujg := ipli^^std)- 
Given A as above, d'^pg = 9g is regarded as a differential equation for a com plex structure 
J on A. The local existence of integrable such structures is shown in [|Gu£ 1 and the same 



strong uniqueness theorem as that stated above is proved; in particular, the locally defined J's 
automatically glue together. Furthermore, dd'^pg = dOg = tOg is Kahlerian, i.e., pg is a strictly 
plurisubharmonic function on every adapted neighborhood A. 

The connection between these two notions of adapted structure is given by the following 
result. 



Theorem 6.2 | ]LS| The 1-form 6g on a domain A equipped with the adapted complex structure 
in the Riemannian sense satisfies d'^pg = 9g. 

As a consequence of the uniqueness theorem, the Riemannian and symplectic notions of 
adapted structure are equivalent. This allows the use of properties of plurisubharmonic func- 
tions, which we now briefiy summarize, in the Riemannian setting. 

Basic properties of plurisubharmonic functions. 

A (smooth) strictly plurisubharmonic function p : A ^ M on a complex manifold is by 
definition a potential of a Kahler form dd'^p — lo. In other words, in holomorphic coordinates 
the complex Hessian H{p) := ^ gf . ^ is positive definite. Equivalently, the restriction p\c to 

every (local) complex curve C is strictly subharmonic in the sense that the Laplacian of p\c is 
negative. On disks in C such functions have the (strong) mean value property. 

Strictly plurisubharmonic functions have strong convexity properties. In fact, holomorphic 
coordinates can be chosen so that p is strictly convex on the underlying real domain. Thus 
the maximum principle holds: A strictly plurisubharmonic function never takes on a (local) 
maximum value. The following also refiects this strong convexity. 

Proposition 6.3 Let p : A — > M he strictly plurisubharmonic. Let M d X be a connected local 
real submanifold such that (1) p has some constant value c on M and (2) c is a minimal value 
of p in a neighborhood of M . Then dimg i\/ < dime A 

This follows immediately from the fact that the complex Hessian of p is positive definite, and 
therefore if F is a real subspace of the tangent space that is isotropic with respect to the real 
Hessian of p , then dimg V ^ dime A. 
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The adapted structure for Riemannian symmetric spaces. 

Here only irreducible Riemannian symmetric spaces Qq = Gq/Kq of negative curvature 
are considered. In other words Go is noncompact and simple. If go = to © Po is a Cartan 
decomposition, then T^Iq — Gq Xa',, Po- Now we recall the Riemannian notion of adapted 
complex structure on domains A in T^Iq. 

Define the polar coordinate map 4" : Tflo ft — G/K by [(go,^)] i-^ go exp(i^).a;o. It is 
well defined and Go ~equi variant. Furthermore, : TpTD,Q — > T^j^jil is an isomorphism at 

points p of the 0-section. 

Let fiadpt be the connected component containing the 0-section of the set where 4' has 
maximal rank, which is {p G Tflo \ is an isomorphism}. Let J be the (integrable) complex 

structure on fladpt which is defined by pulling back the complex structure of by 4*. 

Proposition 6.4 The structure J on fladpt is adapted. 

Proof. Let xq = (a;ojO), the neutral point in TVIq . Identify fio with Go.a^o- Recall that 
the geodesies through ^(xo) are given by l~parameter groups: ^{t) = g e:qi{t£,) .xq for ^ S po • 
Thus S7,(t) = [{g.exp{t^).e,s£,)] and "if o j'^(t + is) = g exp{{t + is)£,)xo- Consequently, for an 
appropriately small disk, 7'''|A : A — > ^adpt is holomorphic. □ 

Proper actions. 

An action L x M ^ M of a, topological group on a topological space is said to be proper if 
the induced map L x M — > M x M , (g, x) '—^ {g{x), x), is proper. Under minimal assumptions on 
the spaces at hand, this can be expressed as follows: For all sequences {g„} C L and {x„} C M 
such that and gn(xn) ^ y there exists a convergent subsequence gn^ ^-^ g in L. All 

isotropy groups of a proper action are compact. 

The Go-action on fio — Gq/Kq is proper, so the Go-action on M — TUq is as well. Although 
^ ■ ^adpt ^ has finite fibers, it does not immediately follow that the Go-action on its image 
is proper. Nevertheless, the Go-isotropy groups in '^{fladpt) are at most finite extensions of 
the corresponding (compact) isotropy groups in fladpt and therefore are themselves compact. It 
would therefore be natural to consider canonically defined neighborhoods of CIq in in which 
the Go-isotropy groups are compact. 

From the point of view of the Riemannian conjugate locus there is a very natural candidate 
for such a domain. See ||^. In order to define it, assume as usual that the Cartan involution 
of Go is the restriction of that for G which in turn defines its maximal compact subgroup G,j. 
Consider the restriction of the polar coordinate map 5" to the fiber T^g^lo = po at the neutral 
point in T^Iq. It maps sufficiently small open neighborhoods of G po diffeomorphically onto 
neighborhoods of the neutral point xq in the G„-orbit G„.a;o = Gu/Kq in Q. 

The maximal such set is determined as follows by the conjugate locus of the invariant metric. 
Let ao be a maximal abelian subalgebra of go which is contained in pQ, Aq be its set of (real) 
roots, and for a € Ao, let Ha be the affine hypersurface G Oo : a(^) = ^}. Define wo to 
be the connected component containing G po of ao \ UaeAo and let Eo := i^o exp(^^o)■ 
Then Go X Ko ^0 is naturally embedded in TD,q as an open neighborhood of the 0-section by 
the action map (5,^) 1— > [(50, C)] and '^{(Goxko^o) ■ ^0 xaTq S 17 is a diffeomorphism onto its 
image (see e.g. |C|). This image ^(Go x^^ Eo) = Go. exp(ia;o).a;o was considered in 
we denote by ^Iag ■ The situation can now be summarized as follows. 

Proposition 6.5 The restriction o/^E' to the open subset GqX Kq^o ofiladpt is a diffeomorphism 
onto its image ^ag', *^ particular the G^-action on ^Iag is proper. Furthermore, VLag comes 
equipped with the ^ -induced, Go-invariant strictly plurisubharmonic function p = pgO vp^i and 
its associated Kdhler form uj := dd'^p. 
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The following result sheds more light on the picture. 



Theorem 6.6 [BHH| The domain Gq ^ Ko maximal domain of definition for the adapted 



complex structure. In particular ^adpt = Gq ^ Ko o*^^ ^ ■ ^adpt ~* ^AG biholomorphic. 

Incidence geometry and the domain ftj . 

Recall that an Iwasawa-Borel subgroup is a Borel subgroup B C G that contains a factor 
AqNq of an Iwasawa decomposition Gq — KqAqNq , where Kq can be any maximal compact 
subgroup of Gq. 

Consider our usual cycle space setup for an open Gg-orbit D in Z = G/Q, where ilw{D) 
is regarded as an open neighborhood of ilo — Gq.Cq = Gq.xq in = G/K. Recall that a 
-B-Schubert variety 5 in Z is defined to be the closure of a i?-orbit O in Z . Write S* = O U F, 
where Y is the (finite) union of _B-orbits on the boundary of O. The following remark motivates 
a number of our considerations. 

Theorem 6.7 IfSnD^d), then codinizS* ^ dime Gq. 

Proof. If 5 n -D 7^ 0, then O n I? 7^ as well. Without loss of generality we may assume that 
Go — KqAqNq, B D AqNq and Kq.zq = Go is the base cycle. In particular, AoA'o.Go = D and 
the real codimension of AqNq.z in D is at most the real dimension of Go for every z £ D. Since 
B D AqNq, it follows that O n Go 7^ and the desired dimension bound holds. □ 

Invariant incidence varieties. Let B be an Iwasawa-Borel subgroup and S a. B- 
Schubert variety with coAmizS — dime Cq = q and 5 H Go 7^ 0. Using the same type of 
argument as above, we make the following observation 

Proposition 6.8 The intersection SnD is contained in O, and the intersection OHCq with the 
base cycle is nonempty and transversal at each of its points. Furthermore, each of its components 
is an AoNQ-orbit. 

Note that the above Schubert varieties are completely determined topologically by the 
Poincare dual PD(Co) in _ff»(Z;Z). In particular, there exist such varieties and given one 
we refer to the components of S* H O as Schubert slices. Using deeper considerations, we have 



the following improvement of the above proposition (see |HW3|) 



Proposition 6.9 Every Schubert slice S intersects every cycle C € VlwiD) in exactly one 
point, and that intersection is transversal. 

Turning to incidence varieties, we consider B as above and let Y be any closed i?-invariant 
subvariety of Z. Then the incidence variety Ay := {C €ri:GnF7^0}isa closed, i?-invariant 
algebraic subvariety of fl. It is a proper subvariety if and only if Go ^ Y. We only consider this 
case. Since B has only finitely many orbits in Z, there are only finitely many candidates for 
Ay. 

Consider the special case Y — S \0, where 5 is a Schubert variety containing a Schubert 



slice S as above. Here, due to Proposition 3.9, we refer to 5 as a transversal Schubert variety. 

Recall that Y has the structure of a very ample Cartier divisor. Let r(S', 0{*Y)) denote the 
space of meromorphic functions on S with poles only on Y and let r{n,0{*Ay) be the analo- 
gously defined space of functions on il. The trace transform Ts : r{S,0{*Y)) r{n,0{*Ay)) 
is defined by Ts{f)(C) = T,pecnsfiP)- dUl^llEl' lH Appendix]). Of course this is first 



defined at the cycles G which intersect S generically. This resulting function in r{n,0{*Ay)) 
arises via analytic continuation. With care about cancellations in the defining sum, one proves 
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Theorem 6.10 [BK] Given C E Ay there exists f e T{S, 0{*Y)) such that the polar set 



'n{Ts{f)) contains C . In particular, Ay is a B -invariant complex hypersurface. 

Incidence varieties in f2 which are hypersurfaces are denoted by Hy and are called incidence 



hyper surf aces. If is a transversal Schubert variety and Y — S\0, then, by Proposition 6.7, 
the incidence hypersurface Hy is contained in the complement of ^w{D) in Q.. 

One of our original goals was to prove that ^wr{D) is a domain of holomorphy in il, i.e., 
that given a divergent sequence {C„} C Vlw{D) there exists a function / e 0{Q.w{D)) with 



lim„^oo |/(C'n| — oo. The following criterion (see, for example, |GuR|), formulated in the setting 
where Q is afHne, is useful for this. 

Theorem 6.11 If for every point C in the boundary hd{Qw{D)) in D, there exist a complex 
hypersurface H in VL which is contained in n\ flw{D) with C £ H , then ^1\y{D) is a domain 
of holomorphy. 

One consequence of our recent work is that for every C S hd{Q,w{D)) there exists an 
incidence hypersurface Hy defined by a transversal Schubert variety ([ HW3[ |, also Section 0). 

Domains defined by invariant hypersurfaces. 

Let _B C G be an Iwasawa-Borel subgroup and H a i3-invariant complex hypersurface in fi. 
The family {g^H} g^s^Q^ consists of all such hypersurfaces which are equivalent in the sense that 
the variation within the family only depends on the choice of B. Since H is Ao-^o^invariant 
for some Iwasawa decomposition Go ~ K^^Ai^Nq, this family is the same as {fco-fflfcoe-ffo- The 
connected component of p|j,^g^^(r2\ fco^^) containing the neutral point xo G is a Go-invariant 
domain defined by H. We denote it VIh • 

If S* is a transversal Schubert variety, we have the associated incidence hypersurface H = Hy 
where Y = S\0 . The connected component V!.s{D) containing xq of the intersection of all 
where H = Hy is an incidence hypersurface associated to such a Schubert variety is referred to 
as the Schubert domain associated to D in VL. 

Finally, the Iwasawa domain is defined to be the connected component containing xq of the 
intersection of all the fin, i.e., as H ranges over all the (finitely many) B-invariant hypersurfaces. 

Proposition 6.12 fin, ^s{D) and f2/ are Go-invariant domains of holomorphy in fl. 

Proof. By definition these domains are Go-invariant, connected open subsets of O. Every 
boundary point of such a domain is contained in a hypersurface koH which is contained in its 



complement in Q. The desired result then follows from Theorem 6.11 □ 
From the definitions, Qj C flsiD) and, if _ff is a Schubert incidence hypersurface then 

The domain flj can be viewed from several different perspectives. For example, it is the 
same as the polar Xq which is defined to be {gxQ G 17 : g G G and Xq C g^o} where Xq is the 



closed Go-orbit in Z ~ G/B and ko is the open iiT-orbit in Z (see |Ba]). R. Zierau remarked 



that Xq = VLi (see jHWaj ). The polar can also be regarded as a type of cycle space (see |GM|| ) 



The equaUty il/ = Q.ag ■ 

As noted above, the domain fi/ has various guises. From the point of view of holomorphic 



extension of certain special functions on Oq, it is shown in |K5] that ^Iag C 17/ for the classical 



groups. From the polar viewpoint it is shown in |Ba| that 17/ C ^ag ■ Equality of these 
domains comes down to the opposite inclusion, VIag C 17/ . That was proved in using the 
plurisubharmonic function p that is defined by the adapted complex structure; see Proposition 
Now we put all this together. 
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Theorem 6.13 0.1 = Oag 



Proof. For the inclusion Oag C il/ let p be the Go-invariant strictly plurisubharmonic function 



of Proposition 6.5. Suppose to the contrary that some hypersurface H which is invariant under 
an Iwasawa-Borel subgroup B has nonempty intersection with Oag or, equivalently, for some 
Iwasawa decomposition Go = KqAqNo and some xi G Oag the orbit AN.xi is not open. 

The orbit AqNq.xq = flo is totally real of half dimension in il. On the other hand, no 
AoiVo-orbit in AN.xi n Oag is of this type. Now p{xo) = and p > along all other Go-orbits. 
For a; G Eo near xq, i.e., for p{x) > sufficiently small, AqNq.x must still be totally real. 

Let r be the smallest value of p|So such that for some xi E {p = r} the orbit AqNq.xi 
is not not totally real. Such a value must exist, because the total reality of an j4oA^o-orbit is 
equivalent to the openness of the corresponding ^TV-orbit; in particular, we might as well let 
Xi be the same point as that which was denoted by xi at the outset. 



Apply Proposition 6.3 to the complex manifold X = AN.xi HflAG, the strictly plurisubhar- 
monic function p\x and the real submanifold M := AqNq.xi. It follows that dima M ^ dime X . 
But this implies that dimR AI ^ dimu f2o ~ 2, contrary to the fact that the AoiVo -orbits in Oag 
all have the same dimension as Hq . 



Our proof of the inclusion flj C Oag follows that in [Ba|. In the flavor of the present 
paper we use the fact that the Go-action on Qj is proper. That follows from the Kobayashi 
hyperboHcity of il/ see Section ^). 

Suppose that there is a sequence {zn} C Oag H flj with z„ ^ z e hd{flAG) H flj . From the 
definition oHIaGj it follows that there exist {(?,„} C G and {x„i} C exp(iCi;o) such that gmi^m) = 
Zjn . Write = kmCimi^m in a KqAoNq decomposition of Go . Since {/cm} is contained in the 
compact group Kq , it may be assumed that fc„j — > fc; therefore that = Om'^m ■ Since luq is 
relatively compact in a, it may also be assumed that Xm — > a; € c£(exp(«wo)). Thus Xm = TmXo, 
where {T^} C exp(iwo) and T. Write amnn{xm) = amnmTmXo = amnmXo, where 

flm = 0-mAm and hm = T~^nmTm are elements of A and iV, respectively. Now {zm} and the 
limit z are contained in f2/ which is in turn contained in AN ■ xo . Furthermore, AN acts freely 
on this orbit. Thus am — > a G A and hm n G N with an ■ xq = z. Since Tm T, it 
follows that am a £ Aq and n,„ — s- n G A^q with an.x = z. Since z ^ Oag, it follows that 
X € bd(exp(ia;o)), and z G f2/ implies that x € Oj . 

On the other hand, since x G bd(exp(zLi;o))j the isotropy group Gx is noncompact. But Qj 
is Kobayashi hyperbolic see Section |9|). Therefore the G-action on flj is proper (see e.g. 
and consequently x ^ flj, which is a contradiction. That completes the proof □ 

7 Transversal Schubert Varieties. 



In this Section we outline the methods introduced in |IIW3| and their main applications, in 
particular the fact that 0\y{D) = ils{D). The intermediate results, which follow from a sort of 
triality, are of complex analytic interest. For example, at every boundary point p G bd(D) we 
construct a, {q + l)-codimensional Schubert variety S with p G S and S' C Z \ D. Due to the 
existence of the family flwiD) of (/-dimensional cycles in D, this exhibits the maximal possible 
degree of holomorphic convexity. It should be underlined that S arises as an extension of a 
complex analytic manifold in the the boundary orbit and therefore its relation to the signature 
of the Hessian of a boundary defining function is unclear. 

The Borel groups B considered here are Iwasawa-Borel subgroups, the ones that contain 
an Iwasawa component AqNq . The Schubert varieties are always those which are closures S of 
orbits O in Z of Iwasawa-Borel subgroups, and Y := S \ O. If codim^S" = dime Co = 9 and 



5 n Go 7^ 0, then S is called a transversal Schubert variety (see Proposition 6.9) 
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As usual D is an open Go-orbit in Z = G/Q and Cq = Kq.zq is the base cycle. Here we 
view i^w{D) C with := G.Co in the cycle space C'(Z). In other words, in the case when 
= K we do not replace O by the finite cover G/K even if the G-stabilizer of Go properly 
contains K, and we do not exclude the hermitian holomorphic case where Q — G.Gq is an 
associated compact hermitian symmetric space. 
Duality. 

Throughout this subsection 7 £ OrbzGo (resp. k G OrhzK) denotes one of the finitely 
many Go-orbits (resp. if-orbits) in Z. The first example of duality was proved at the level of 



open Go-orbits |W2]: Every open Go-orbit 7 contains a unique compact K -orbit k. 

Let us reformulate this in a way that makes sense for all Go- and if-orbits. For this first 
observe that, since Gq O K = Kg, the intersection 7 n k of any two Gq- and X-orbits is Kq- 
invariant. 

For z e 7 n K we refer to the orbit Kq.z as isolated if it has a neighborhood J7 in 7 so that 
kOU — Kq.z. Finally, let us say that (7, k) is a dual pair if 7 n k contains an isolated i^o-orbit. 

Matsuki's duality theorem, which is an extension of the above statement for open orbits, 
states that there is a bijective map : Orb^Go — > OrhzK such that (7, k) is a dual pair if and 
only if K = /i(7) Q. 

In the original version, duality was defined by 7nK being nonempty and compact, but it was 
soon realized that this is equivalent to the condition that 7 H k be a single Xp-or bit. Th e mo re 



recent proofs, which involve the Morse theory related to a certain moment map (|MUV], |BL]), 
use the above weaker notion. However, in the end, if an intersection 7 H k contains an isolated 
Kfj-orbit, then it is a Kq -orbit, and the intersection 7 n k is transversal along that orbit. 

In our work we use the following non-isolation property which is implicit in the proofs in 



| MUV| and gL| 



Proposition 7.1 // (7, k) is not a dual pair, then every Kg-orbit Kq.z injHK is contained in 
a Ko-invariant locally closed submanifold Af C 7 H k with dimg M = dimg Kq.z -f 1. 



This symplectic approach also yields information about the topology of the Go-orbits HW3[ | 



Proposition 7.2 // (7, k) is a dual pair, but 7 H k 7^ 0, then the K^-orbit j H k is a Kq- 
equivariant strong deformation retract ofj. 

Triality. 

Here we describe a sort of triality where, in addition to the orbits 7 and k above, we 
incorporate Schubert varieties of Iwasawa Borel subgroups B. For this we fix an Iwasawa 
decomposition Go — KqAqNo and B containing AqNq. For k S OrhzK let c£{k) denote its 
closure in Z and define 5^ to be the set of all i3-Schubert varieties S such that codim^S* = 
dime K and S D c£{k) ^ 0. 

The Schubert varieties of a fixed Borel subgroup generate the integral homology of Z and 
consequently is determined by the topological class of c£{k); in particular, it is non-empty. 

The following can be regarded as a statement of triality. 

Theorem 7.3 // (7, k) is a dual pair, then the following hold for every S € S^. 

1. S D c£(k) is contained in ^ H k and is finite. If x € S H k, then {AN){x) — B[x) — O, 
where S = cl{0), and S is transversal to k, at x in the sense that the real tangent spaces 
satisfy T^(S) ® T^in) = T.^{Z). 

2. The set T, = £(7, S, x) :— AqNq(x) is open in S and closed in 7; in particular it is a locally 
closed complex submanifold of Z. 
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3. Let c£{T,) and 0^(7) denote closures in Z . Then the map Kq x c£(S) --> c£{j), given by 
{k,z) ^—^ k{z), is surjective. 

Proof. Let X E S n c£{k). Since = { + a + nis the complexification of the Lie algebra version 
00 = ^o + ao + no of Gq = KqAqNo , weh.aveT^{AN{x))+T^{K{x)) = T^iZ). As x G S = c£{0) 
and AN C B, we have dim AN (x) ^ dimi?(a;) ^ dimO = dim 5*. Furthermore a; € c£{k). Thus 
dimi^(a;) ^ dim^c. If x were not in k, this inequahty would be strict, in violation of the above 
additivity of the dimensions of the tangent spaces. Thus x G k and Tx{S)+Trc{K) — Tx{Z). Since 
dimS* + dimK = dimZ this sum is direct, i.e., T^{S) © T^{k) = T^iZ). Now also dim. AN (x) = 
dimS* and dimK{x) = dimK. Thus AN{x) is open in 5*, forcing AN{x) = B{x) = O. We have 
already seen that K(x) is open in k, forcing K(x) = k. For assertion 1 it remains only to show 
that 5* n K is contained in 7 and is finite. 

Denote 7 — Gq{x). If 7 7^ 7, then (7 , k) is not dual, but 7 H k is nonempty because it 
contains x. By the non-isolation property ( |7.l|) , we have a locally closed i^To-invariant manifold 
M C 7 n K such that dimM = dimifola;) + 1- We know T^iS) ® T,,{k) = T^{Z), and K{x) = 
K, so T^iAoNoix)) n Tx{M) ^ 0. Thus T^{AoNo{x)) + T^{Ko{x)) has codimension 1 in the 
subspace Tx{AqN{){x)) + Tx{M) of T!r(7), which contradicts Go — KqAqNq . We have proved 
that {S n c£{k)) C 7. Since that intersection is transversal at x, it is finite. This completes the 
proof of assertion 1 . 

We have seen that T,{AN{x)) ® T,{K{x)) = r,(Z), so T,{AoNq{x)) ® T,{Kq{x)) = T,{j), 
and Gq{x) = 7. From the basic properties of a dual part, in particular the transversality of the 
intersection 70 K, we have dim yloiVo(a;) = dimra;(7) — dimra;(«;n7) = dimTa;(Z) — dimTa;(K) = 
dim^iV(a;) — dim 5. Now AoNo{x) is open in S. 

Every AoA^o-orbit in 7 meets Ko{x) because 7 = Go{x) = AoNoKo{x). By the transversality 
of the intersection 7 n k, every such AoiVo-orbit has dimension at least that of S = AoNo{x). 
Since the orbits on the boundary of S in 7 would necessarily be smaller, it follows that E is 
closed in 7. This completes the proof of assertion 2. 

The map iiTo x E ^ 7, by {k, z) ^ k{z), is surjective because KoAqNq{x) = 7. Since ii'o is 
compact and 7 is dense in 0^(7), assertion 3 follows. □ 

Let us now indicate the construction given in [ HW3| of the supporting Schubert variety at 



each boundary point p g bd(£'). For this it is convenient to ref er to a Go-orbit in bd(D) as 



generic if it is open in bd(_D). The transversality property [HW3| also gives us 



Lemma 7.4 If ^ is a generic orbit in hd{D), k is dual to 7 and S = c£{0) € Sk, then 
codim^S' > dime Go. 

There are increasing sequences {Ok} of B-orbits with Co = O, dim Ok = dim O + k and 
Ok C c£{Ok+i), for fc ^ n — dimcO, so the above Schubert variety can be enlarged to obtain 
the following consequence of triality. 

Theorem 7.5 For every p S bd(_D) there exists an Iwasawa-Borel subgroup B <Z G and a 
B-Schubert variety S with (1) codim^S = dime Go + 1 and (2) p G S and S <Z Z \ D . 

Proof. Given a point p in a generic orbit 7, we may take an appropriate conjugate of the 



Iwasawa-Borel subgroup of Theorem 7.3 to obtain S satisfying all of the required conditions 



except that it may be too small. In that case we enlarge it to be {q -\- l)-codimensional by the 



above procedure. By Proposition 6.7 this Schubert variety is also contained in Z \ _D. 

If p G bd(D) is not generic, then nevertheless it is the limit p„ — > p with 7 — Go.pn generic 
and with _B„-Schubert varieties Sn which have the desired properties. 
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Since {5'„} is contained in a compact space of cycles and {Bn} can be regarded as a sequence 
in the (compact) minimal Go-orbit, by going to subsequences, Sn S and Bn B, where S 
is a _B-Schubert variety with the desired properties. □ 

The equality ilwiD) = QsiD) . 

This equality is equivalent to the following statement. 

Theorem 7.6 Given C G ]:>d{Qw{D)) there exists a transversal Schubert variety S = O U Y 
withCnY ^9. 



We outline a constructive proof of the existence of such an S. See [HW2] for the details. 

For any initial p G C n hd{D) we begin with a smallest i?-Schubert variety S such that 
(i) Sod 7^ 0, (ii) p is in a component of F = S \ O that is contained in the complement 
Z \ D, and (iii) all components of Y which contain p are themselves in Z \ _D. Define S ^ hy 
codim^^ = dime +1 — S. If J = 0, then there is nothing to prove. 

Now let (5 > 0. Let U denote the union of the components of Y which have empty intersection 
with D and V be the union of the remaining components. V is non-empty by the minimality 
assumption on S. 

Using intersection theory information, roughly speaking the fact that the intersection Cq.U 
is zero in homology, one proves that S can be replaced by a lower-dimensional B-Schubert 
variety Si which is contained in V with p being replaced by pi £ C H bd(-D). 

Continuing in this way, one eventually determines a transversal Schubert variety Sg and a 
point PS € C C\ bd(-D) which is also contained in Ys. □ 

Corollary 7.7 Vlw{D) VLs{D) 

Proof. Given C E hd{il.w{D)), the above result yields an incidence hypersurface Hy which 



contains C and which is contained in the complement fl \ ^w{D); see Theorem 6.1C. □ 
Corollary 7.8 The cycle space flw{D) is a Stein domain in fl. 



Proof. When fl is affinc, this follows immediately from Theorem 6.11 and the equality flwiD) — 
fls{D). In the hermitian holomorphic cases, where f2 is the associated compact symmetric space, 
note that, since Pic(ri) = Z, it follows that the complement of any Hy in O is affine. Since 
flwiD) C (il \ Hy), the result follows for the same reasons as above. □ 

Spaces of cycles in lower dimensional Go orbits. 



We recall the setting of |GM|] . For Z = G/Q, 7 G Orbz(Go) and k e OrhziK) its dual, let 
G{7} be the connected component of the identity of {g G G : g{K)nj is non-empty and compact }. 
Note that G{7} is an open iiT-invariant subset of G that contains the identity. Define C{j} := 
G{'y}/K. Finally, define C as the intersection of all such cycle spaces C{j} as 7 ranges over 
Orbz(Go) and Q ranges over all parabolic subgroups of G. 

Theorem 7.9 C = Qag ■ 



This result was checked in [ GM | for classical and hermitian exceptional groups using case by 
case computations, and the authors of | GM|] conjectured it in general. As will be shown here, 



it is a consequence of the fact that Vtw{D) = Vts{D) in the special case where D is an open 
Go-orbit in G/B and of the following general result | |GM , Proposition 8.1]. 



Proposition 7.10 [i^D^c/B open ^w{D)) C C 
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Proof of Theorem. The polar Xq in Z = G/B coincides with the cycle space Czi'^o), where 
7o is the unique closed Go-orbit in Z. As was shown above, this agrees with the Iwasawa domain 
fli which in turn is contained in every Schubert domain QsiD). Thus, for every open Go-orbit 
Dq in Z ^ G/B we have the inclusions 

\ ' 'DCG/ B open / 

Intersecting over all open Go-orbits D in G/B, the equalities 

(n nw{D)] =C = ni = (C] ^w{D)] 

y IdcG/B open ^ > j \\ \dcG/B open ^ ^ J 

are forced, and C = Qag is a consequence of 51/ — ^Iag- D 



8 Cycle Domains in the Hermitian Case. 



In this section Go is a group of hermitian type, in other words B — Gq/Kq is a bounded 
symmetric domain. We give a concrete description of flw{D). As indicated in Section ^, either 
il,w{D) = B 01 B, or G.Cq = is affine. Thus it is enough to consider the latter case. At first 
we will replace flwiD) by the connected component containing xq in its preimage in G/K and 
denote the latter by fl. 

Choosing a system of roots in the usual way, we regard the bounded symmetric domain B 
of Go as the Go-orbit of the neutral point xq G X = G/P- and its complex conjugate as the 
orbit of the analogous point G AT = G/P+. We view zq = (xq,1xq) as the base point for 
f2 = G.zo = G/K ^ X X X, i.e., the open G-orbit by its diagonal action. We identify B x B 
with its image under the natural embedding B x B ^ X x X and note that this lies in ^l. 



The following is proved by a reduction to the polydisc case [BHH 



Proposition 8.1 Qag = B x B 



In ||WZl|] it was shown that flwiD) C B x B. Thus Corollary |7/7| together with Theorem 

|6.13| imply the following characterization. 

Theorem 8.2 ([EIW3], [ WZ3| ) If Gq is of hermitian type, then either (i) fl is the compact dual 
symmetric space to Gq/Kq and flwiD) is B or B, or (ii) O = G/K is affine with K/K finite 
and flwiD) = B x B. In case (ii) ^w{D) lifts bijectively to G/K. 

Proof. It is enough to consider the case where 51 is affine. As we have seen above, after lifting to 
n = G/K, nw{p) C BxB = nAG- But, by Theorem Hag = 51/ , and nwiD) = fls(il) 



by Corollary \l.7\ Since by definition 51/ C 51s(_D), the result follows at the level of G/K. 
Furthermore, B x B is a cell. Thus it agrees with its image in G/K by the canonical finite 
covering map (see Proposition 10.20). □ 



Remark 8.3 This theorem can also be proved using results from | ]GM| . 



9 Kobayashi Hyperbolicity. 

The Kobayashi pseudometric dx is defined on any complex manifold X; see Section |]. If it is 
a metric, i.e., dK{x,y) > if a; 7^ y, then X is said to be Kobayashi hyperbolic. One checks 
that it is a metric on bounded domains and vanishes identically in the case where X, e.g., is 
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the complex plane. Since liolomorphic maps are Kobayashi-distance decreasing, if there exists 
a non-constant holomorphic map f : C X, then X is not Kobayashi-hyperbolic. 

With this in mind it is useful to introduce a weaker notion of hyperbolicity: X is said to be 
Brody hyperbolic if there are no non-constant holomorphic maps f : C X. 

If X is Kobayashi hyperbolic, then its group AMio{X) of holomorphic automorphisms, 
equipped with the compact-open topology, is a Lie group acting properly on X . This group is 
in general not semisimple, but nevertheless semisimple subgroups act properly. 

Proposition 9.1 Let G be a semisimple Lie group with finitely many components such that 
has finite center. Then any smooth almost effective action of G by holomorphic transformations 
on a hyperbolic manifold X is proper. 

Proof. Let L := Ant{X)'~'. By hypothesis on G and its action on X, the action has finite kernel, 
so for the proof we may assume that G is connected and contained in L. 

Now let R be the radical of L and i = i? • S* be a Levi-Malcev decomposition, where S is 
semisimple. The corresponding Lie algebra decomposition is [ = r xi s. Let T := R O S . It is a 
discrete normal (thus central) subgroup of S. Projection L L/R = S/T =: S, followed by the 
adjoint representation of S, is a Lie group morphism 93 : i — > Ad{S) C GL{5), and Ker((/3|(5) is 
a discrete central subgroup of G, thus finite. By construction ^p{G) is a semisimple subgroup of 
GL{5), thus closed in GLis). As Ker((/3|G') is finite now G is closed in L. Since the action of L 
on X is proper, it follows that the action of G is also proper. □ 

Families of hypersurfaces. 

It is a classical result that the complement of the union of (2m + l)-hyperplancs in general 
position in CPm is Kobayashi hyperbolic Q . Let us make the notion of general position precise 
in a context which is appropriate for our applications. 

Since the complex manifolds which we consider are embedded in projective spaces by sections 
of line bundles, it is natural to regard a "point" as being in the projectivization P(F*) of the 
dual space of a complex vector space and a "hyperplane" as a point in ¥{V). We regard a subset 
S C P(F) as parameterizing a family of hyperplanes in P(y*). A non-empty subset S C P(V") 
is said to have the normal crossing property if for every fc g N there exist Hi , . . . LI^ e 5 so that 
for every subset / C {1, . . . , fc} the intersection Hie/ |/|-codimensional. If |/| ^ dime ^ ^ 

this means that the intersection is empty. 

In the sequel {S) denotes the complex linear span of S in P(l^), i.e., the smallest complex 
subspace in V{V) containing S. If {S) = V{V) we say that S* is a generating set. 

Proposition 9.2 A locally closed, irreducible real analytic subset S with (S) = V{V) has the 
normal crossing property. 

Proof. We proceed by induction over k. For k = 1 there is nothing to prove. Given a set 
{Hs^ , . . . , Hsi^ } of hyperplanes with the normal crossing property and a subset / C {si, . . . , s^}, 
define 



We wish to prove that S\C£k 7^ 0- For this, note that each 7i (/) is a real analytic subvariety of S. 
Hence, if S' = C£k, then S = n{J) for some J with Aj ^ 0. However, {H G P(F*) : H D Aj} 
is a proper, hnear plane £.{J) of F{V). Consequently, S C C{J), and this would contradict 
(S) = P(F). Therefore, there exists s G S* \ C£k, or equivalcntly, {Hsi, ■ ■ ■ , Hg^, Hs} has the 



A/:=n H 



H{I) ■.= {seS:HsD A/} C4 := |J 



JC{si,..,Sfc},Aj#0 



H{J). 



normal crossing property. 



□ 
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In the case of finitely many hyperplanes in CPm the condition that Hi,...,H2m+i are 
in general position is equivalent to their having the normal crossing property, in which case 
CP(y*) \\jH.j is Kobayashi hyperbolic (g], or see @, p. 137]). 

Corollary 9.3 If S is a locally closed, irreducible and generating real analytic subset oJV{V), 
then there exist hyperplanes iJi, . . . i?2m+i G S such that the complement P(F*) \ is 
Kobayashi hyperbolic. 



Our main application of this result arises in the case where S is an orbit of the real form at 
hand. 

Corollary 9.4 Let G be a reductive complex Lie group, Gq a real form, V* an irreducible G- 
representation space and S a Go-orbit in V{V). Then there exist hyperplanes Hi, ... , H2m+i G S 
so that V{V*) \ U Hj is Kobayashi hyperbolic. 

Proof. From the irreducibility of the representation V* , it follows that V is likewise irreducible 
and this, along with the identity principle, implies that (S) = P(y). □ 

Invariant hyperbolic domains in il. 

We begin by briefly discussing the formalism for equivariantly embedding = G/K in a 
complex projective space. In order to avoid discussions of infinite-dimensional spaces, we let 
X be a G-equi variant, smooth, projective algebraic compactification of Q. Since X is rational 
and we may assume that G is simply connected, every line bundle L ^ X is a G-bundle; in 
particular there is a canonically induced action on the space T(X,h) of sections. 

Now let B be an Iwasawa-Borel subgroup of G and H he a, i3-invariant complex hypersurface 
in il. Since H is algebraic, it is Zariski open in its closure c£{H) in X. 

Let L ^ X he the line bundle defined by c£{H) and s E T{X,L) the defining section. Since 
c£{H) is B-invariant, s is a B-eigenvector. 

Define V to be the irreducible G-representation subspace of r{X, L) that contains s. Let 
ip : X ^ P{V*) denote the canonically associated G-equivariant meromorphic map. 

Proposition 9.5 Assume that Gq is not of hermitian holomorphic type. Then the restric- 
tion (p\q : — !■ P(y*) is a G-equivariant, finite-fibered regular morphism with image a quasi- 
projective G-orhit in P(V^*). 

Proof. Since f is G-equivariant and its set of indeterminacies is therefore G-invariant, the 
restriction to the open G-orbit is base point free. The fact that (^|n is finite-fibered is a 
consequence of being non-constant, e.g., s is not G-fixed, and the fact that the G~isotropy 
group Gxo — K in dimension-theoretically a maximal subgroup of G. (It is here that we use the 
assumption that Gq is not of hermitian holomorphic type.) □ 
We are now in a position to prove the main theorem of this section (|^, |FII]). 



Theorem 9.6 If Go is not of hermitian holomorphic type, B <Z G is an Iwasawa-Borel sub- 
group, and H is a B -invariant hypersurface in = G/K , then fin is Kobayashi hyperbolic. 

Proof. We replace (p by its restriction to and only discuss that map. By definition every 
section t G V is the pull-back <^*(t) of a hyperplane section. Thus, there is a uniquely defined 
B-hypersurface H in V{V*) with ip~'^{H) = H. Let C ¥(¥*) be defined analogously to Qh, 
i.e., = F{V*) \ UgeGo Applying Corollary ^ to F(y*) and S -.^ Go ■ H d V{V), it 

follows that the domain 17^ is Kobayashi hyperbolic. Furthermore, the connected component of 
(^^^(rj^) which contains the base point xo is just the original domain fin. Since holomorphic 
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maps are distance decreasing and </> : fin is locally holomorphic, it follows that fin is 

also Kobayashi hyperbolic. □ 

Let us summarize what we have presented up to this point. 

Summary 9.7 If Gq is of hermitian holomorphic type, then either U is the compact hermitian 
symmetric space dual to the bounded symmetric domain B = Gq/Kq, and either ilwiD) — 
^s{D) = B or fl\Y{D) — ils{D) — B. If Go is of hermitian nonholomorphic type, then is 
afRne and 

= ni = ns{D) = nwiD) = bxb. 

If Go is not of hermitian type, then, with the usual convention that Qw{D) C O = G/K, 

for H any complex hypersurface which is invariant by an Iwasawa-Borel subgroup. All of the 
domains are Go-invariant, Stein and Kobayashi hyperbolic and therefore the Go-actions are 
proper in every case. 



10 The Maximal Domain of Hyperbolicity. 

Recall the basic sequence of inclusions 

(10.1) nAG = cnw{D) = ns{D) 

for any complex hypersurface H <Z — G/K which is invariant under an Iwasawa-Borel sub- 
group of G, when D is not of hermitian holomor phic type. All of these domains are Go-invariant, 
Stein and Kobayashi hyperbolic; see Summary |9.7|. 



Our goal here is to outline the proof of the following main theorem of |FB]. 

Theorem 10.2 The only Gt^-invariant, Stein, Kobayashi hyperbolic domain in £7 which con- 
tains ^AG is VLag itself. 



This implies that the sequence (10.1) of inclusions is a sequence of equalities in the non- 
hermitian case. With the classification in the hermitian case (Section ||), this yields the following. 

Theorem 10.3 In the hermitian holomorphic case, VLw{D) is the bounded domain B or B 
associated to Go ■ In all other cases, Oag = 0/ = ViwiD) = QsiD)- 



The proof of Theorem 10.2 involves three main steps: (1) Understanding the invariant theory 
of the Go-action on fl; in particular, the orbit structure on hd{flAG)- (2) For every generic 
boundary point z, determining an sZ2-triplc defined over M so that the orbit S.z — Q2 intersects 
Qag in a- 2-dimensional polydisk which is the ^Iag for the group S. (3) Proving Theorem 10.2 
in the case of S* = SL2{C). 



All details which are omitted in the following sketch can be found in [ FH | . To be consistent 
with the notation of that paper, we let x € SI be a general point and xo G f2 the chosen base 
point with Gx^ = K . 

The linear model. 

Using a classical linearizing map (see [|Ml| ) we Go-equivariantly embed O in a linear space 
where Jordan decomposition can be used in an optimal way. 

The basic map. Let a denote complex conjugation of g over go • Write r for the complex- 
linear extension to g of the Cartan involution of go , so r is the holomorphic involution of g 
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with fixed point set t. We extend conjugate-linearly to g, so 9 becomes the conjugate-linear 
involution of g whose fixed point set is the compact real form g„ , in other words 9 becomes a 
Cartan involution of g. 

Define rj : G AutR(g) by 77(2;) — ao Ad{x) or oAd{x~^). We regard a, r and G as operating 
on AutR(0) by conjugation, the action of G being via Ad. Let N denote the normalizer of K in 
G. The basic properties of ij are as follows. 

• ry is right A^-invariant and defines an embedding G/N ^ AutR(g). 

• ry is equivariant with respect to the left action of Gq on G. 

• The image Im(77) is closed, tr-invariant and r-invariant, a(r](x)) ~ ri(x)^^ and t(?7(x)) — 
r]{T{x)). 



To see that Im(?7) is closed we use a basic lemma of invariant theory (see [Hu, p. 117]) 



Lemma 10.4 Let V be a finite dimensional real vector space, H a closed reductive algebraic 
subgroup of GLr{V) and s g GL]gi{V) an element which normalizes H. Regard H as acting on 
GL]^{V) by conjugation. Then, for a semisimple s the orbit H.s is closed. 

To prove that Im(77) is closed it is enough to show that G.r = {Ad{g)TAd{g)~^ : 5 € G} is 
closed in Autc(0). Since r is semi-simple and normalizes G in this representation, this is then 
immediate. 

From now on, unless otherwise stated, we view 77 as a map rj : —^ AutR(g) which is is 



essentially a diffeomorphism onto its closed image. In this language Lemma 10.4 also leads to 
Proposition 10.5 If ri{x) ^ s is semi-simple, then Gq.x is closed. 



Proof. It is enough to show that Gq.s is closed. By Lemma 10.4 the complex orbit G.s is 
closed. Define a : AutE(g) — > AutE(g) by d{ip) — {a{Lp))'^ . Here a acts by conjugation as 
usual. Note that Im(?7) belongs to the fixed point set Fix((T). Since G.snFix((T) consists of only 
finitely many Go-orbits [ |Br| , it follows that Gq.s is closed □ 

Jordan decomposition. Let rj{x) = u ■ s denote the Jordan decomposition of an element 
in Im(r/). Then the unipotent factor u e AutR(0)°, and u = exp(ad(j^)) = Ad(exp(i^)) for some 
nilpotent G Qo ■ If (/? G AutR(0) let q"^ denote its fixed point set. The fact su = us can be 
expressed ly S . Also, a{r]{x)) — ri{x)~'^ implies t] g igo . Now compute ri{exp{^i').x) = s. In 
summary we have the following result, describing a lifting of the Jordan decomposition. 

Proposition 10.6 For a; G with Jordan decomposition ri{x) — u ■ s there exists a nilpotent 
element € n igo such that u = Ad(exp(z/)) and ri{exp{^i') ■ x) — s. 

Orbit structure. 

Here we outline some basic information on the orbit structure of the Go-action on Q. The 
main objective is an understanding of the Go-action on bd{QAG)- 

Closed orbits. In any invariant theoretic situation it is of central importance to move in 
a systematic way from a point in a non-closed orbit to a closed orbit in its closure. Here we 
accomplish this by means of special s^2-triples. 

Since cr(s) = and s is semisimple, is a cr-invariant reductive subalgebra of g. Let 
g = t) ffi q be its cr-decomposition. Now apply the Jacobson-Morozov Theorem. 

Lemma 10.7 Let e Cz Cl ig be nonzero and nilpotent. There exists an 5l2-triple (e, h, f) in g" 
{i.e., [e, /] = h, [h, e] — 2e, [h, f] — —2/) such that e, / e q and ft. G f). 
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Using this sZ2-triple we are able to explicitly move to a closed orbit. 



Proposition 10.8 If ri{x) — us is the Jordan decomposition, then the orbit Gn.ri{x) = Go.(su) 
contains the closed orbit Gq.s C Ini(?7) in its closure ci{GQ.ri{x)). In particular, Go-ri{x) is 
closed if and only if rjix) is semi-simple. 



Proof. Let u = Ad(expi') with as in Proposition 10. 6| . Hence, by Lemma 10.7 there is a 



sZ2-triple {v,h,f ) (e = v) such that \th,v] — 2tv, i.e., Ad(exp(tft.))(i^) — e^^v for every t g 
Note also that exp(IRiJ) C Go n cxp(g^) by construction of the s/2-triple. It follows that 

r]{ex.Tp{th) ■ x) = exp(th).{us) — Ad{exp{th)) ■ us ■ Ad(exp(— i/i)) = 
= Ad(exp(ti7)) • u ■ Ad(exp(-tff)) • s = 
= Ad(exp(ti/))Ad(expi^)Ad(exp(-iiJ)) • s = 
= Ad(exp(e^V)) • s. 

For t —00 it follows exp{tH).{us) = Ad(exp(e^*z^)) ■ s ^ s. Hence, the closed orbit G.s 
lies in the closure of G.{us). In particular Go.{us) is not closed if u 7^ 1, i.e., if ri(x) is not 



semisimple. This, together with Proposition 10.5 implies that G.rj{x) is closed if and only if 



rj{x) is semi-simple. Recall that the image Im(77) is closed. This forces s G lra{rf} and the proof 
is now complete. □ 

Elliptic elements. An element a; € f2 is called elliptic if 77 (x) is elliptic in the sense that 
it is semisimple and all its eigenvalues have absolute value 1. Let Q,m denote the set of elliptic 
elements. 

If a: e G„ , so 9{x) = x, then 9ri{x) = ri{x)9, so ri{x) is elliptic. Therefore Gu-Xq C ^eii- 
Since Qeii is Go-invariant now Gq. exp(iao).a;o C i^eii- The opposite inclusion follows via classical 
methods. We have proved 

Proposition 10.9 fteii = Go- exp(mo.a;o) 

The following is a key ingredient for understanding the Go-orbit structure in hd{QAG)- 

Proposition 10.10 exp(ioo).xo n c£{V,ag) = c£{eyip{iujo) -Xq) 

Proof. If X e c£{exp{iujo) .xo) , then it is elliptic and therefore its orbit Go-a; is closed. In other 
words exp(zao).a;o H c£{Q,ag) ^ c^(exp(zwo).a:o). 

For the opposite inclusion, observe that if s, s' e exp(jao).a;o and s' S Gq.s, then s' — ko{s) 
for some element fco of the Weyl group. Thus, if s G c^(exp(«ci;o)-Xo), then s' G c£(exp(zu;o).a;o) 
as well. Therefore, in order to prove the opposite inclusion it is enough to show that, given 
s' € exp(mo.a;o) n c£{^ag), there exists s e c€(exp(ia;o)-a;o) with s' g Go-s. 

Given s' as above, there exist sequences {sn} C exp(ia;o)-a;o and {s'„} C i^AG such that 
s'„ € Go.s„, s'^ s' and s„ s G c€(exp(«a'o).xo). Consider the (real) categorical quotient 
map TT : AutR(0) — > AutK(0)//Go . It is continuous, the base is Hausdorff and in every fiber 
there is exactly one closed Go-orbit. Since 7r(s„) = 7r(s^), it follows that Go-s = Gq.s'. □ 

Corollary 10.11 Let flee denote {x : Gq.x is closed }. Then 

flci n c£{flAG) = Go.c£(exp(iu;o)-a;o) = ^eii n d{flAG)- 



Proof. From Proposition 10. 9| , ^Iag C fleii- By continuity, the semi-simple part of r]{x) 



is elliptic for every x G c£{D,ag)- Thus Qce H c£{ D,ag) C fleii n c£{il,AG), because elements 
of closed orbits are se mi-sim ple. Proposition [10. 9| gives ilci H c£{ftAG) C Go.c^(exp(iwo).a;o), 
and from Proposition 10.10| it follows that Go.c£{exp{iuJo)-Xo) C flee H c£{ftAG)- So we have 



flci n c£(flAG) C Go-c£{exp{iLL!o) .xq) = fleii n c£{flAG)- Finally, if x G fleii, then in particular it 
is semi-simple and G.a; is closed. This proves the remaining inclusion. □ 
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Existence of a (32-slice. 

HiLBERT Lemma. We have fixed ao . Let G^.y C hd{VLAG) be a non-closed orbit. We 
determine a base point z € Go -J/ with 77(2) = u ■ s such that the point xi corresponding to s is 
in bd(exp(ja;o).a;o). That uses the nilpotent element v associated to u, and then an sZ2-triple 
{v — e, h, /) defined over M with z in the orbit S.xi of the corresponding complex group. We 
regard this as an analogue of the Hilbert Lemma in the case of actions of reductive complex Lie 
groups. 

Lemma 10.12 Every non-closed Go-orbit in hd[QAG) contains a point z — exp(j/) •exp(i^) -xq 
where ^ € bd(a;o) C Oo and where v e g''(oxp(«?)) q jg nonzero and nilpotent. 



Proof. Let rj{y) ~ su be the Jordan decomposition and let v ^ C] igo as in Proposition |10.6 
Then r]{y) — ?7(e xp(— ^t^) exp(-^iy) • y) — Ad(expi/) o ?7(exp(ii') ■ y) — u - s. By Proposition 



10.5 



10.9 



and Proposition 10.9 the semisimple element ri{exp{^i^) ■ y) is elliptic. Hence, Proposition 
implies the existence of g G G and ^ g bd(a;o) such that exp(ii^) • y = exp(i^) • xq . Define 
e := Ad(g)(— i;/). Then g ■ y — exp(e) exp(i^) • y = exp(e) exp(i^) • xq. Finally, e e Q^'^ = 
gr,(exp(»4)) ^ Lemma |l0.12| is proved. □ 



Let e be as above and S be the complex group determined by the above s/2-triple. Direct 
calculation shows that 5*^!^ = C*. Thus the orbit S.xi is equivariantly biholomorphic to either 
the 2-dimensional affine quadric or to the complement of a smooth quadric curve in CP2. The 
former is naturally realized as the complement of the diagonal in CPi x CPi and the latter as its 
quotient by the Z2-action defined by reversing its factors. At the level of homogeneous spaces 
the former is S/T^ where T is the complexification of a (compact) maximal torus Tq, and the 
latter is S/N , where N is the normalizer of T in S. 

In order to remind the reader of the connection to the quadric, the orbit S.xi is referred to 
as a (52-slice whenever its intersection with VLag is an U,ag associated to the real form 5*0 of S 
which is defined by the restriction of ct, the complex conjugation of G over Go . 

Genericity. Starting with an arbitrary Go-orbit on hd{^lAG) it is difficult to determine 
how the S-orbit S.xi intersects ^ag ■ However, for Go-orbits of generic boundary points this 
will be relatively straightforward. Here a point z € hd{flAG) is called generic if Go. 2 is not 
closed and if the point xi — exp(i^)a:o constructed above by the Hilbert Lemma is a smooth 
point of hd{e'xp{iujo) .xq) . 

Recall that bd(a;o) is defined by hyperplanes. We refer to the points in hd{exp{iu!o) .xo) 
which correspond to points which are contained in two or more hyperplanes as corners. Let £ 
denote the Go-saturation of the set of such corners, i.e., points z of hd{nAG) such that Go-^ 
has a corner in its closure. Let C be the set of points z G hd{ilAG) such that Go. 2 is closed, 
so C = Go.hd{exp{iLUo) .xq) . Finally, let hd{QAG)gen be the closure of the complement of f U C 
in hd{flAG)- It follows that this is contained in the set of generic points in the above sense. A 
careful look at the invariant theory for the Go-action, leads to the following density result. 

Proposition 10.13 The set hd{flAG)gen is open and dense in hd{U,AG)- 

Using detailed knowledge of the Go-isotropy groups at the smooth points of bd(exp(iwo). 2^0)1 
one proves the desired result on the existence of (52-slices. 

Proposition 10.14 At every generic boundary point there exists a Q2-slice. 

Given y e hd[QAG)gen we of course move it via Go to an optimal point 2 in Go.y so that 
S.z contains the smooth boundary point xi € bd(exp(jajo)-2;o) as above. 
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Analysis of a (32-slice. 

Let S = SL2(C) and let So = 5-^2 (ffi) bo embedded in 5* as the subgroup of real matrices. As 
usual J^o = S02{^) and K = S 02(C). Dq and -Doo denote the open S'o-orbits in CPi containing 
the respective Ko-Hxed points and 00. 

S acts diagonally on Z = CP^ x CP^ with one open orbit O, the complement of the diagonal 

diag(CP'^) in Z. fl is the complex symmetric space S/K. Note that in CP'^ x CP"'^ there are 
4 open SL2{M.) x S'L2(K)-orbits: the bi-disks Da x Df} for any pair (a,/3) from {0,oo}. As 
So-spaces, the domains Dq x and D^c x Dq are equivariantly biholomorphic; further, they 
are subsets of il, and the Ricmannian symmetric space Sq/Kq sits in each of them as the totally 
real orbit 50(0,00) or 5o(oo,0). Each can be considered as the Qag associated to S and the 
real form So' 

^AG = Do X = So ■ expit^o • (0, 00) or x Do = Sq ■ expiti^o • (00, 0). 

We choose the first: luq = (^f ■ j)ha where /;,„ G a is the normalized coroot, i.e., a'{h„) = 2. 

Our main point here is to understand ^o-invariant Stein domains in which properly contain 
flAG- By symmetry we may assume that said domain has non-empty intersection with Dq x Dq. 
Observe that (Dq x Do) n = Dq x Dq \ diag(Z?o)i and other than diag(_Do) all Sq orbits in 
Dq X Dq are closed real hypersurfaces. Up £ DqxDo^ diag(Do) let r2(p) be the domain bounded 
by So -p and diag(£'o)- We shall show that a function which is holomorphic in a neighborhood 
of Sq- p extends holomorphically to fl{p). 

S := {(— s, s) : < s < 1} C Do X Dq is a geometric slice for the action of So on Dq x Dq . 
We say that a (1-dimensional) complex curve C C C Z is a supporting curve for bd(ri(p)) 
at p if C n ci{fl{p)) = {p}. Here ci{fl{p)) denotes the topological closure in Dq x Dq . 

Proposition 10.15 If p G Dq x Dq \ diag(£>o) there is a supporting curve for hd{Q{p)) at p. 

Proof. We consider Dq C C as the unit disc. We need only construct a supporting curve 
C C at each point ps = (— s, s) G S, s 7^ 0. Define Cs := {(— s + z,s + z) : z G C}. To prove 
Cs n c£(n(ps)) = {ps} let d : Dq x Dq ^ M be the distance function of the Poincare metric of 
Dq . It is an S'o-invariant, and its values parameterize the S'o-orbits on Dq x Dq . 

We now claim that d{—s + z, s + z) ^ d{—s, s) = d{ps) for ^; e C and {—s+z, s+z) G DqX Dq, 
with equality only for z = 0, i.e., Cs touches c£(fl(ps)) only at ps- To prove this, we compare 
the Poincare length of the Euclidean segment seg(2; — s,z + s) in Dq with that of seg(— s,s). 
Writing the corresponding integral for the length, it is clear from a glance at the integrand that 
d{—s + z,s + z) > d{—s, s) for 2; 7^ 0. This completes the proof □ 

From the above construction the boundary hypersurfaces S{p) are strongly pseudoconvex 
from the viewpoint of diag(£'o)- The smallest Stein domain containing a S'o-invariant neighbor- 
hood of Sq{p) is fl{p) \ diag(£'o)) so the following is immediate. 

Corollary 10.16 If p G Do x Dq \ diag(£'o) f is holomorphic on a neighborhood of Sq ■ p 
then f extends holomorphically to fl{p) \ diag(£)o). Ifp G D^o x D^o \ diag(£>oo) the analogous 
statement holds. 

The set of generic boundary points is the union of the two S'o-orbits, bdge„(-Do x Doo) ~ 
(bd(£'o) x Doo) U {Do x bd(Doo)). Let z G bd(Do) x D^o (or 2; G Do x bd(Doo), respectively). 

Corollary 10.17 Let n C Q2 C CP^ x CP^ be an So-invariant Stein domain that contains 
Dq X Doo and its boundary point z. Then fl also contains {Dq x CP^) \ diag(CP^) (or also 
contains (CP^ x Dqo) \ diag(CP^), respectively). 
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Proof. Let B he a ball around z which is contained in ft. For p E B{z) n (Do x £'o) sufficiently 



close to z, 5o ■ <z C for all q G B{z) n {Dq x Dq). The result follows from Corollary 10.16. □ 
If Vl is as in Corollary 10.17, then the fibers of the projection of £7 C CP"'^ x 



to the first 



factor CP^ can be regarded as non-constant holomorphic curves / 



SI. In particular, 



Corollary 10.18 If VI is as in Corollary 10.17, then Q is not Brody hyperbolic. 



Characterization of cycle domains. 

In the previous sections, in order to avoid unnecessary notation, we have often replaced 
^w{D): component of Co in {gCo \ g E G and gCo C D} C G/K 
by its finite cover 

^w{D): component of IK in {gK \ g € G and gCa C D} C G/K. 

Proposition 10.20| below, shows that in fact the covering G/K G/K is bijective on VLw{D), 

inducing a holomorphic dilfeomorphism ^w{D) —> ^w{D) and justifying the above-mentioned 
replacement. For our main theorem, however, even though we have not yet come to the proof 
of Proposition 10. 2C , we still consider Q.w{D) as sitting up in G/K . 



Theorem 10.19 Either we are in the hermitian holomorphic case and VLw{D) is B or B, or 

nAG = ^i = ^w{D)^ns{D). 



HW3 or 



Proof. If Gq is of hermitian type, then the result is contained in Theorem B.2, or sec 
iWZ3|| . Other wise we have the inclusions and equalities ( |l0.l[ ) from Summary |9.7| . By Theorem 
10. 2| above, VIh — ^ag-, and consequently all of those inclusions are equalities. □ 



Finally we view the cycle space as it really is, and verify that the standard projection tt : 
G/K G/X restricts to a holomorphic diffeomorphism of r2vi/(£') o\iioQ.w{D). The projection 
TT is given as identification under the right action of the finite group F = K\K on G/K. F 
permutes the components of {gK \ g E G and gG^ C D} = Tr~^{{gGo \ g G G and gGg C D}). 

Thus il.w{D) is the quotient of ^}w{D) by its stabilizer in F. 



By Theorem 10.19 , flw{D) = ^Iag- Since Qag is a cell, a finite group of diffeomorphisms can 
act freely on it only if it is trivial. Thus one may indeed regard flwiD) as being in SI = G/K. 

We summarize this as follows. 

Proposition 10.20 The restriction tt : flwiD) flwiD) of the projection G/K G/K is 
biholomorphic. In particular, Qw{D) is a cell. 



The following Corollary is contained in IIW3 |. Also see | H W4 1 . 
Corollary 10.21 In all cases, flwiD) is a contractible Stein manifold. 
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Part III: Applications and Open Problems. 



In this Part we go more closely into applications of the complex geometric methods described 
and developed in Part II. 



11 Recent Results on the Double Fibration Transform. 

We continue the discussion of double fibration transforms from Section ^, taking advantage of 
the material just described in Part II. 

As explained above in Proposition |6.9| , there is an Iwasawa decomposition Gq = KqAqNq 
such that the Schubert slice S := ^0^0(2:0) C D meets every cycle C £ ^lw(D) transversally in 
a single point within D. That gives a map 

(11.1) (/) : QwiD) ^ E := AaNa{za) hy C ^ {Y. n C) e D. 

Note that (j>~^{z) consists of all cycles C e flwiD) that contain z, so (f)~^{z) = fj,~^{z) —: F. 

Let Jo denote the isotropy subgroup of ^0-^0 at 1 and let Fq = ii~^(zq). Note that Jq acts 
on Fq . Realize ( p_l.l ) as the AgA'o-homogeneous fiber bundle {AqNq) xjoFq^ AqNq/Jo . The 



subsets ilw{D) and Fq := {C G \ zq G C} are semialgebraic in fl, so their intersection Fq 
has only finitely many topological components. As ^wiD) is simply connected, it follows that 
S ~ AqNq/Jo is a solvmanifold with finite fundamental group. Thus S is C"^-diffeomorphic to 



a cell. Hence the fibration (11.1) is trivial. By Proposition 10.20, its total space is a cell. So 



now the base and total space of (11.1) are cohomologicaly trivial, and thus the same holds for 
the fiber F. We have proved 



Theorem 11.2 [[IW4] Let F denote the fiber of the holomorph ic fi bration fi : T{D) D. Then 
F is connected and H^{F; C) = for all r > 0. In particular ( [4.4| ) is satisfied for every q, and 
the double fibration transforms P : H'^{D:E) —f H^{M]TZ'>{^* E)) are injective for all sufficiently 
negative E, —^ D. 



Remark 11.3 HW4 | The fiber space projection : QwiD) ^ S is the restriction to open 
subsets of a holomorphic bundle projection cj) : AN Xj Fq — > O, as follows. Let Fq {C E 
fl \ zq £ C}. The complex submanifold O = B{zq) C Z where B is a Borel subgroup of G 
that contains AqNo ■ Thus S = O n D is open in O by the discussion of Schubert cells and 
Schubert slices in Theorem [7. 3| above. A and N are the respective complexifications of ^0 and 
iVo . T{D) is open in AN{Fo), which is the total space of <j> : AN XjFq^O. The connection 
with (j) : ^lw{D) ^ S is that flw{D) = AoNo{Fo), which is open in AN{Fo = ANxjFq), where 
J is the isotropy subgroup of AN at zq ■ Since it is the restriction of (j), the map (p : QwiD) S 
is holomorphic. 



Now we have adequately addressed the injectivity requirement (4.13) for the double fibration 



transform of a flag domain, and we turn to the question (4.14) of its image. Since the Stein 
manifold ^lw{D) is contractible, every holomorphic vector bundle E flwiD) is holomorphi- 
cally trivial, and in particular the Leray derived bundles = W{G, /i*(E)|,^-i(c)) over il.w{F)) 
are holomorphically trivial. Here we have two requirements for ( [1.14[ ): we need 

(11.4) a canonical choice of holomorphic trivialization of E^ QwiD), and 

(11.5) an explicit (in that trivialization) system of PDE that specifics the the image of P. 

This is work in progress. 
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12 Unitary Representations of Real Reductive Lie Groups. 



In this Section we look at some of the imphcations of the double fibration transform for repre- 
sentations of real reductive Lie groups. 

Harish-Chandra's analysis of the holomorphic discrete series can be viewed from the per- 
spective of the double fibration transform as follows. Let Go be of hermitian type, B — Gq/Kq . 
I n thi s case, of course, D = B = T{D) — flwiP) , the double fibration transform is the identity, 
( 11.4 ) is completely standard, and the system (|ll.5 ) consists of the d operator. Let — > B 
denote the homogeneous holomorphic hermitian vector bundle associated to the representation 
Ex of Kq of highest weight A. By use of his system of strongly orthogonal noncompact positive 
roots, and the explicit holomorphic trivialization of Ea B, he proves (i) a holomorphic section 
oi Ex ^ B is L'^{B) if and only if its /fo~isotypic components are L'^{B), (ii) if some nonzero 
i^^o~isotypic holomorphic section of Ea ^ S is (B) then the constant section fx , value equal 
to the highest weight vector of Ex , is L'^{B), and (iii) fx is L'^{B) if and only if {X + p,f3) < 
where p is half the sum of the positive roots and /? is the maximal root. 

Narasimhan and Okamoto |N0] extended the Harish-Chandra construction to "almost all" 
discrete series representations of a real group Go of hermitian type, again always working over 
D = B = T{D) = Uw{D) where the double fibration transform is more or less invisible. 

The double fibration transform first became visible, at least in degenerate form, in Schmid's 
holomorphic construction of the discrete series ([^^, |35|). There Z = G/B for some Borel 
subgroup B and D = Gq/Tq where Tq is a compact Cartan subgroup, To C Kq C Gq . Only the 
"real form" ip : D ^ Gq/Kq of the double fibration appears: there Gq/Kq appears instead of 
the cycle ilw{D); correspondingly D appears instead of the incidence space 1{D). Injectivity 
of this real double fibration transform : 7f«(D;E) ^ H°{Gq/ Kq]¥)) is given by Schmid's 
"Identity Theorem". That theorem says that, under appropriate restrictions, a Dolbeault class 
[bj] £ H'^{D;E) is zero if and only if every restriction oj is cohomologous to zero on every fiber 
of : D Gq/Kq. This was extended a bit by Wolf [W3|, for flag domains of the form 
D = Gq/Lq with Go general reductive and Lq compactly embedded in Gq . 

The double fibration transform first appeared in modern form in the paper [WeW| of Wells 
and Wolf on Poincare series and automorphic cohomology. The only restriction there was that 
D = Gq/Lq with Lq compact, and a small extension of the Identity Theorem was used to, in 
effect, prove injectivity of the double fibration transform. 

The Penrose transform applies to the case D = SU{2,2)/S(U{1) x C/(l,2)). There Lq is 
noncompact, and perhaps that is the first such case to be studied carefully. See |BE]. Back- 
ground work on interesting flag domains with noncompact isotropy includes, of course, parts 
of Berger's classification [Be] of semisimple symmetric spaces. Wolf's study |W1] of isotropic 
pseudo-riemannian manifolds, and of course | |W2[ |. Important cases of construction of unitary 
representations using double fibration transforms on flag domains with noncompact isotropy 
were studied in Dunne-Zicrau [DZ] and Patton-Rossi PR2| . This area was first studied sys- 
tematically in Wolf-Zierau [WZ2|. 

Finally, as noted in [W6|, there are indications of a strong relation between the double fibra- 
tion transforms of | WZ2| and the construction of unitary representations by indefinite harmonic 
theory of (Rawnsley, Schmid & Wolf pSW[). 



13 Variation of Hodge Structure. 

In this Section we indicate the connection between Griffiths' theory of moduli spaces for compact 
Kahler manifolds (period matrix domains and linear deformation spaces), on the one hand, and 
fiag domains, cycle spaces and double fibration transforms on the other hand. Along the way 
we will sketch some relevant aspects automorphic cohomology theory as developed by Wallach, 
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Wells, Williams and Wolf. 



For Griffiths' theory see [Grl] and [Gr2]. There are expositions contained in |Gr3|, p^ , 
I Wei I, and |We2|. The ffi'st Hodge-Riemann bilinear relation specifies a complex flag manifold 
Z — G/Q and the second Hodge-Riemann bilinear relation specifies an open Gg-orbit D C Z. 

Let X denote a compact Kahler manifold, Hq{X;C) and Hq{X;R) the complex and real 
spaces of primitive cohomology classes in degree r, and Hq{X;C) — J2p+q=r ^o'"^ i-^'^'^) 
decomposition by bidegree. This specifies the Hodge filtration C i^^ C • • • C F^) of 
iJ5(X;C), where = E»<^ -^o"'''!^; C), and thus the complex fiag = (F" C i^^ C • • • C 

F") where u is the integer part of (r— 1)/2. We have a nondegenerate bilinear form 6 on Hq{X; C) 
given (on Dolbeault representative differential forms) by 6(^,77) = (— J ui"--^ A ^ A 77. 

Here w is the Kahler form of X. Evidently b{HP'\X ; C) , H^' {X ; C)) = unless p + p' = 
r = q + q'. Define w{^) (v^Y^'^^ for ^ e iJ«'P(X;C). One can formulate the Hodge- 
Riemann bilinear relations as (1) b pairs Hq''^{X; C) with its complex conjugate Hq^{X; C) and 
(2) h{(^,ri) :— 6(w^,?7) is positive definite on Hq{X;C). 

If r is even, say r — 2t, then b is symmetric. It is positive definite on Hq^^'''{X;C) ® 
Hq^~\X; C) for i < t, negative definite on Hq*{X; C). The (identity component of the) isometry 
group of {HQ{X;C),b) is the complex special orthogonal group G — SO{2h + fc;C) where k = 
dimiJQ'*(X; C) and h = X]i<t with hi — dimi?Q ~*'*(X; C). The dimension sequence of the fiag 
^{X) specifies the complex flag manifold Z — G/Q consisting of all the flags £ = [E^ C C 
• • • C F*-^) in H^{X]C) with b{E*-'^ , E*'^) = 0. The (identity component of the) isometry 
group of {Hq{X; M), b) is the identity component Gq = SO{2h, fc)" of the real special orthogonal 
group SO{2h, k). The second bilinear relation above shows that the isotropy subgroup Lq of Gq 
at T{X) is compact. It follows that Lq is of the form {U{ho) x • • • x U{ht-i) x SO{k). The fiag 
^{X) ranges (as X varies) in the open Go-orbit 



D = {£\b:>Oon E'-^ + F*-i} ^ S0{2h, k)/{U{ho) x • • • x U{ht-i) x SO{k)). 
ReveUih,) preserves {H^^'''{X;C)+H'„'''''{X;C))nH^{X;R), and SO{k) preserves iJ*'*(X; M). 

If r is odd, say r = 2< — 1, then b is antisymmetric, so Hq{X;C) has even dimension 2m 
and the isometry group of {HQ(X;C),b) is the complex symplectic group G = Sp{m;C). The 
dimension sequence of the flag J-{X) specifies the complex flag manifold Z = G/Q consisting of 
ah the flags £ = (F° C F^ C • • • C F*) in H^{X; C) with b{E\ F*) = 0. The isometry group of 
(iJo(X;K),6) is the real symplectic group Go — Sp{m;R). As above, Gq has compact isotropy 
subgroup Lq at ^{X), necessarily of the form U{ho) x • • • x U{ht)- The flag T{X) ranges (as X 
varies) in the open Go-orbit D ~ {£ \ b nondegenerate on each (7Jq^*'*(X; C) +i/Q '^^'(X; C))}, 
which is realized as Sp{m]'R) / {U {ho) x • • • x U{ht)) where hi = dimiJQ^*'*(X; C) as before. 

Since Go has compact isotropy subgroup Lq on D, we have Lq C Kq , and the holomorphic 



double flbration (4.19) is supplemented by maps D = Gq/Lq Gq/Kq C flwiD). 

Choose a basis {71, . . . , 7t,} of the space Hr{X; Z) / (torsion) of r-cycles on X. Given J-'{X) we 
have a basis {w^, . . . , w"} of Hq°{X; C), then Hq~^'^{X; C), continuing through the 6-isotropic 
space of the flag ^{X). That defines a u x v period matrix 

U{X) := 




which of course specifies ^{X). As in the case of period matrices of Riemann surfaces, one can 
change the basis {7^} by any integral element of Go and change the basis {w-* } by any element of 
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Go that docs not change ^{X). Thus the moduh space for r-forms of compact Kahler manifolds 
X with given Hodge numbers Hq''' := dim Hq'''{X; C), p + q ~ r, is the arithmetic quotient 

T\D = Gz\Go/Lo 

(13.1) = 50(2/1, fc; Z)\SO{2h, k)/{U{ho) x • • • x U{ht_i) x SO{k)) for r even, 

= Sp{m;Z)\Sp{m;R)/{U{hQ) x ••• x Uih)) for r odd, 

where ft-i = ^o~''* ^'^^ ^ = Gz is defined by the lattice Hr{X;Z) in ffr.(X;K). A variation of 
Hodge structure of X corresponds to a deformation of Kahler structure of X, that is a fiber 
space ip : U ^ V and a distinguished point vq V such that the Xy = ip~^{v) are compact 
Kahler (or algebraic) manifolds, X — X^^ , with /ig''(X„) = /ig''^(X), and such that the Xy vary 
holomorphically (or algebraically). That defines a holomorphic map of — > T\D. 

Classically one constructs automorphic functions on T\D as quotients of F-invariant sections 
of holomorphic line bundles over D (automorphic forms of a given weight). Also classically D is 
a bounded symmetric domain Sp{g; R)/U{g) and one works in a fixed holomorphic trivialization 
of the line bundles over D, so the F-invariance condition is expressed by a transformation law. 
In this way one constructs the function field of the moduli space r\D. 

The classical theory of automorphic functions must be modified in our context because in 



general D has no nonconstant holomorphic functions [W2|, and in general nontrivial homo- 
geneous vector bundles over D have no nonzero holomorphic sections. Instead one considers 
sufficiently negative homogeneous holomorphic vector bundles K ^ D. Roughly speaking, those 
are the bundles whose cohomology, and whose sheaf cohomology, viewed as Go-modules, 
have the same underlying Harish-Chandra module. Their cohomology occurs in degree dim Go 
where Gq = Kq / Lq ■ One looks for automorphic cohomology, meaning F-invariant classes in 
H''{D;0(E)). That is a bit remote from the idea of a function field for T\D, but the double 
fibration transform P : H'i{D;0{K)) H°[VLw{D)]0{¥))) and the holomorphic trivialization 
of VLw{D) carry the automorphic cohomology space H'^{D; 0{E))^ to a space of holomor- 

phic functions ^lw{D) H'^{Co;0{E\cq)) with a certain transformation law under F. In this 
sense T\^}w{D) can be a good replacement for F\D as universal deformation space. 

In much of the literature one considers only the situation where Go is of hermitian type and 
the bounded symmetric domain B = Gq/Kq is used instead of ilw{D). (Of course they are 
the same if D is of hermitian holomorphic type.) When Gq is not of hermitian type then again 
Gq/Kq is used instead of ilw{D), and it is considered somewhat of an obstacle that Gq/Kq is 
not a complex manifold. Our use of T\ilw{D) addresses this point. 



In connection with construction of automorphic cohomology. Wells | Wel | showed by direct 



computation that is a Stein manifold in one particular case (r = 2). That result 



was extended in Wells- Wolf |WeW| to the more general situation of open Go-orbits D of the 



form Gq/Lq with Lq compact, using a special case of the double fibration transform together 



with somewhat general methods of complex analysis (Andreotti-Grauert |AnG|, Andreottf 



Norguet ||AN| , Docquier-Gra uert ||DG| ] ) associated to questions of holomorphic convexity and 
the Levi problem. The goal of fWeW]] was construction of automorphic cohomology as convergent 
Poincare ?9-series dric) := ^ry^r^*(^) where c S H'^{D] 0(E)) is a Ko-iimte cohomology class. 
The relevant estimates were derived from semisimple representation theory, specifically from 



Hecht-Schmid |HSc| and Schmid |S4|, and the passage between D and flwiD). 

This theory of Poincare "iJ-series and automorphic cohomology later was developed quite a 
bit. According to [ |W5[ , if F is any discrete subgroup of Go , E ^ £) is sufficiently negative 
and 1 ^ p ^ oo then every F-invariant LP(F\D) class in _ff'(Z?; 0(E)) can be realized as a 
Poincare series z9r(c) where c € H'^{D; 0{E)) is U'{D). In particular this is close to the idea of 
catching all of the function field. The "sufficiently" part of the "sufficiently negative" condition 



on E ^ D is relaxed in Wallach-Wolf |WaW| by construction of an appropriate reproducing 
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kernel. Finite dimensionality of automorphic cohomology was proved by Williams (|Wil|, Wi2|] , 
|Wi3|), using index theory of Moscovici and Connes, for the case where r\D is compact. Despite 
this development, automorphic cohomology has not yet been effectively applied to variation 
of Hodge structure. We expect that the new information on the double fibration transform, 
presented above, will make a difference here. 
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